Selective Dry Cow Therapy: Cow-level analysis
#Mac
knitr::opts_knit$set(root.dir = "~/Dropbox/R backup/SDCT - R")
#Windows
#knitr::opts_knit$set(root.dir = "C:/Users/rowe0122/Dropbox/R backup/SDCT - R")
library(knitr)Load data
load(file="Baseline.Rdata")
load(file="SDCTCOW.Rdata")
load(file="SDCTCOWDHI.Rdata")
SDCTCOW = SDCTCOW %>%
mutate(Tx = recode(SDCTCOW$Tx,
"0" = "Blanket", "1" = "Culture", "2" = "Algorithm"))
Import file BL (cows included for analysis) for descriptive statistics
library(readr)
BL <- read_csv("BL.csv", col_types = cols(X1 = col_skip()))Missing column names filled in: 'X1' [1]head(BL)Inspect data
print(summarytools::dfSummary(BL, valid.col=FALSE, graph.magnif=0.8, style="grid"), method = "render")NAs introduced by coercionData Frame Summary
BL
Dimensions: 1211 x 10Duplicates: 0
| No | Variable | Stats / Values | Freqs (% of Valid) | Graph | Missing | ||||||||||||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | Site [character] | 1. CA 2. IA 3. MN 4. NY |
|
 |
0 (0%) | ||||||||||||||||||||||||||||||||||||||||||
| 2 | FARMID [numeric] | Mean (sd) : 4 (1.5) min < med < max: 1 < 4 < 7 IQR (CV) : 2 (0.4) |
|
 |
0 (0%) | ||||||||||||||||||||||||||||||||||||||||||
| 3 | Tx [numeric] | Mean (sd) : 1 (0.8) min < med < max: 0 < 1 < 2 IQR (CV) : 2 (0.8) |
|
 |
0 (0%) | ||||||||||||||||||||||||||||||||||||||||||
| 4 | Age [numeric] | Mean (sd) : 47 (15.4) min < med < max: 30.4 < 44.5 < 122.3 IQR (CV) : 22.6 (0.3) | 682 distinct values |  |
0 (0%) | ||||||||||||||||||||||||||||||||||||||||||
| 5 | Parity [numeric] | Mean (sd) : 1.9 (0.8) min < med < max: 1 < 2 < 3 IQR (CV) : 2 (0.4) |
|
 |
0 (0%) | ||||||||||||||||||||||||||||||||||||||||||
| 6 | DOSCC [numeric] | Mean (sd) : 4.4 (1.2) min < med < max: 1.6 < 4.3 < 8.6 IQR (CV) : 1.6 (0.3) | 401 distinct values |  |
0 (0%) | ||||||||||||||||||||||||||||||||||||||||||
| 7 | DODIM [numeric] | Mean (sd) : 325 (45.8) min < med < max: 252 < 306 < 584 IQR (CV) : 49.5 (0.1) | 179 distinct values |  |
0 (0%) | ||||||||||||||||||||||||||||||||||||||||||
| 8 | PrevCM [numeric] | Min : 0 Mean : 0.1 Max : 1 |
|
 |
0 (0%) | ||||||||||||||||||||||||||||||||||||||||||
| 9 | PrevSCCHI [numeric] | Mean (sd) : 5.4 (1.3) min < med < max: 2.9 < 5.3 < 9.2 IQR (CV) : 1.8 (0.2) | 629 distinct values |  |
0 (0%) | ||||||||||||||||||||||||||||||||||||||||||
| 10 | DPlength [numeric] | Mean (sd) : 55.7 (7.9) min < med < max: 30 < 56 < 87 IQR (CV) : 9 (0.1) | 54 distinct values |  |
0 (0%) |
Generated by summarytools 0.9.3 (R version 3.6.0)
2019-07-02
Descriptive statistics of subjects at enrollment
Comparison of demographics for each treatment group at dry-off
library(table1)
table1(~ Age + DOMY + DOSCC + PrevSCCHI + factor(PrevCM) + factor(Parity) | Tx, data=Baseline)| 0 (n=1568) |
1 (n=1592) |
2 (n=1544) |
Overall (n=4704) |
|
|---|---|---|---|---|
| Age | ||||
| Mean (SD) | 47.2 (14.6) | 46.9 (15.9) | 46.1 (15.0) | 46.7 (15.2) |
| Median [Min, Max] | 44.7 [30.8, 104] | 44.1 [30.4, 111] | 44.5 [30.4, 122] | 44.4 [30.4, 122] |
| DOMY | ||||
| Mean (SD) | 26.7 (8.87) | 27.9 (8.81) | 27.3 (8.38) | 27.3 (8.70) |
| Median [Min, Max] | 27.7 [4.54, 49.4] | 29.0 [1.81, 49.4] | 29.0 [2.72, 49.4] | 28.6 [1.81, 49.4] |
| DOSCC | ||||
| Mean (SD) | 4.45 (1.21) | 4.38 (1.23) | 4.45 (1.22) | 4.43 (1.22) |
| Median [Min, Max] | 4.33 [1.61, 8.35] | 4.23 [1.61, 8.59] | 4.39 [1.61, 8.25] | 4.32 [1.61, 8.59] |
| PrevSCCHI | ||||
| Mean (SD) | 5.37 (1.30) | 5.49 (1.23) | 5.40 (1.29) | 5.42 (1.28) |
| Median [Min, Max] | 5.23 [2.87, 9.21] | 5.37 [3.14, 9.21] | 5.30 [2.94, 9.21] | 5.30 [2.87, 9.21] |
| factor(PrevCM) | ||||
| 0 | 1344 (85.7%) | 1360 (85.4%) | 1352 (87.6%) | 4056 (86.2%) |
| 1 | 224 (14.3%) | 232 (14.6%) | 192 (12.4%) | 648 (13.8%) |
| factor(Parity) | ||||
| 1 | 616 (39.3%) | 732 (46.0%) | 660 (42.7%) | 2008 (42.7%) |
| 2 | 516 (32.9%) | 416 (26.1%) | 488 (31.6%) | 1420 (30.2%) |
| 3 | 436 (27.8%) | 444 (27.9%) | 396 (25.6%) | 1276 (27.1%) |
Descriptive statistics of subjects at dry-off
Note that this table excludes cows (n=32) that were not included cow-level analysis (eg. cows with long or short dry periods).
table1(~ Age + DOMY + DPlength + DOSCC + PrevSCCHI + PrevCM + Parity | Tx, data=SDCTCOW)| Blanket (n=407) |
Culture (n=410) |
Algorithm (n=394) |
Overall (n=1211) |
|
|---|---|---|---|---|
| Age | ||||
| Mean (SD) | 47.5 (15.0) | 47.2 (16.2) | 46.3 (15.1) | 47.0 (15.4) |
| Median [Min, Max] | 44.8 [30.8, 114] | 44.4 [30.4, 119] | 44.6 [30.4, 122] | 44.5 [30.4, 122] |
| DOMY | ||||
| Mean (SD) | 26.8 (8.88) | 28.0 (8.89) | 27.3 (8.38) | 27.4 (8.73) |
| Median [Min, Max] | 27.7 [4.54, 49.4] | 29.0 [1.81, 49.4] | 29.0 [2.72, 49.4] | 29.0 [1.81, 49.4] |
| DPlength | ||||
| Mean (SD) | 55.3 (7.89) | 55.9 (7.61) | 55.9 (8.16) | 55.7 (7.88) |
| Median [Min, Max] | 56.0 [33.0, 84.0] | 56.0 [33.0, 84.0] | 56.0 [30.0, 87.0] | 56.0 [30.0, 87.0] |
| DOSCC | ||||
| Mean (SD) | 4.46 (1.22) | 4.39 (1.22) | 4.45 (1.21) | 4.43 (1.22) |
| Median [Min, Max] | 4.36 [1.61, 8.35] | 4.23 [1.61, 8.59] | 4.39 [1.61, 8.25] | 4.33 [1.61, 8.59] |
| PrevSCCHI | ||||
| Mean (SD) | 5.40 (1.32) | 5.50 (1.22) | 5.40 (1.29) | 5.43 (1.28) |
| Median [Min, Max] | 5.23 [2.87, 9.21] | 5.37 [3.14, 9.21] | 5.30 [2.94, 9.21] | 5.30 [2.87, 9.21] |
| PrevCM | ||||
| 0 | 350 (86.0%) | 351 (85.6%) | 345 (87.6%) | 1046 (86.4%) |
| 1 | 57 (14.0%) | 59 (14.4%) | 49 (12.4%) | 165 (13.6%) |
| Parity | ||||
| 1 | 158 (38.8%) | 185 (45.1%) | 167 (42.4%) | 510 (42.1%) |
| 2 | 133 (32.7%) | 108 (26.3%) | 123 (31.2%) | 364 (30.1%) |
| 3 | 116 (28.5%) | 117 (28.5%) | 104 (26.4%) | 337 (27.8%) |
Groups appear to be well balanced before and after exclusions, indicating that randomization was successful, and that drop-out is unlikely to affect our assumption of exchangeability between treatment groups.
Comparison of herds
library(table1)
table1(~ Tx + Age + DOMY + DPlength + DODIM + DOSCC + PrevSCCHI + PrevCM + Parity | FARMID, data=SDCTCOW)| 1 (n=76) |
2 (n=82) |
3 (n=263) |
4 (n=388) |
5 (n=242) |
6 (n=47) |
7 (n=113) |
Overall (n=1211) |
|
|---|---|---|---|---|---|---|---|---|
| Tx | ||||||||
| Blanket | 28 (36.8%) | 27 (32.9%) | 93 (35.4%) | 122 (31.4%) | 81 (33.5%) | 16 (34.0%) | 40 (35.4%) | 407 (33.6%) |
| Culture | 27 (35.5%) | 24 (29.3%) | 87 (33.1%) | 136 (35.1%) | 84 (34.7%) | 16 (34.0%) | 36 (31.9%) | 410 (33.9%) |
| Algorithm | 21 (27.6%) | 31 (37.8%) | 83 (31.6%) | 130 (33.5%) | 77 (31.8%) | 15 (31.9%) | 37 (32.7%) | 394 (32.5%) |
| Age | ||||||||
| Mean (SD) | 48.7 (16.0) | 46.8 (14.3) | 46.8 (13.4) | 47.5 (16.7) | 47.0 (15.9) | 42.9 (11.2) | 46.6 (16.2) | 47.0 (15.4) |
| Median [Min, Max] | 45.2 [30.6, 90.2] | 44.6 [30.7, 95.7] | 44.8 [30.4, 95.5] | 44.6 [31.7, 122] | 44.7 [30.4, 115] | 43.7 [31.0, 82.6] | 44.6 [31.0, 114] | 44.5 [30.4, 122] |
| DOMY | ||||||||
| Mean (SD) | 23.0 (6.30) | 26.6 (9.64) | 29.5 (6.91) | 30.3 (7.84) | 19.6 (8.55) | 30.7 (4.81) | 30.6 (6.33) | 27.4 (8.73) |
| Median [Min, Max] | 22.7 [9.07, 38.1] | 28.6 [2.72, 44.0] | 30.4 [8.62, 47.6] | 30.4 [2.72, 49.4] | 20.0 [1.81, 39.9] | 30.8 [16.8, 40.8] | 30.8 [15.4, 48.5] | 29.0 [1.81, 49.4] |
| DPlength | ||||||||
| Mean (SD) | 53.2 (11.8) | 52.5 (5.63) | 57.8 (6.39) | 55.6 (6.49) | 55.7 (8.16) | 60.3 (7.19) | 53.0 (10.6) | 55.7 (7.88) |
| Median [Min, Max] | 52.0 [32.0, 85.0] | 53.0 [30.0, 63.0] | 58.0 [35.0, 87.0] | 56.0 [35.0, 74.0] | 55.0 [35.0, 85.0] | 60.0 [47.0, 84.0] | 55.0 [33.0, 77.0] | 56.0 [30.0, 87.0] |
| DODIM | ||||||||
| Mean (SD) | 346 (51.4) | 334 (52.7) | 314 (37.9) | 325 (44.4) | 329 (51.2) | 328 (43.4) | 321 (39.1) | 325 (45.8) |
| Median [Min, Max] | 326 [292, 584] | 304 [293, 512] | 293 [266, 475] | 296 [283, 468] | 308 [252, 512] | 314 [258, 444] | 307 [272, 476] | 306 [252, 584] |
| DOSCC | ||||||||
| Mean (SD) | 4.54 (1.04) | 4.08 (1.15) | 4.00 (0.907) | 4.47 (1.16) | 5.18 (1.39) | 4.34 (0.966) | 3.91 (1.09) | 4.43 (1.22) |
| Median [Min, Max] | 4.61 [2.56, 7.59] | 3.96 [2.30, 8.17] | 3.98 [2.60, 6.68] | 4.48 [1.61, 8.59] | 5.13 [1.61, 8.35] | 4.47 [2.56, 6.61] | 3.78 [2.56, 7.38] | 4.33 [1.61, 8.59] |
| PrevSCCHI | ||||||||
| Mean (SD) | 5.48 (1.31) | 5.06 (1.31) | 4.77 (1.08) | 5.77 (1.20) | 6.13 (1.11) | 5.02 (0.927) | 4.73 (1.17) | 5.43 (1.28) |
| Median [Min, Max] | 5.31 [3.43, 9.21] | 4.78 [3.33, 9.21] | 4.61 [2.87, 8.21] | 5.65 [3.00, 9.21] | 6.07 [3.40, 9.21] | 5.09 [3.09, 7.06] | 4.53 [2.94, 8.00] | 5.30 [2.87, 9.21] |
| PrevCM | ||||||||
| 0 | 62 (81.6%) | 81 (98.8%) | 228 (86.7%) | 337 (86.9%) | 191 (78.9%) | 46 (97.9%) | 101 (89.4%) | 1046 (86.4%) |
| 1 | 14 (18.4%) | 1 (1.2%) | 35 (13.3%) | 51 (13.1%) | 51 (21.1%) | 1 (2.1%) | 12 (10.6%) | 165 (13.6%) |
| Parity | ||||||||
| 1 | 31 (40.8%) | 33 (40.2%) | 92 (35.0%) | 180 (46.4%) | 104 (43.0%) | 23 (48.9%) | 47 (41.6%) | 510 (42.1%) |
| 2 | 15 (19.7%) | 28 (34.1%) | 95 (36.1%) | 101 (26.0%) | 73 (30.2%) | 17 (36.2%) | 35 (31.0%) | 364 (30.1%) |
| 3 | 30 (39.5%) | 21 (25.6%) | 76 (28.9%) | 107 (27.6%) | 65 (26.9%) | 7 (14.9%) | 31 (27.4%) | 337 (27.8%) |
Outcome 1: Clinical mastitis
Kaplan Meier curve & log rank test for clinical mastitis
Kaplan Meier curve
library(ggplot2)
library(survminer)
KM <- survfit(Surv(CMTAR, CM) ~ Tx, data = SDCTCOW)
knitr::opts_chunk$set(fig.width = 800, fig.height = 900)
ggsurvplot(KM, data = SDCTCOW, title = "", pval = T, conf.int = F,risk.table.col = "Tx",risk.table = T, risk.table.y.text.col = TRUE , surv.plot.height = 5, legend.labs = c("Blanket","Culture","Algorithm"), tables.theme = theme_cleantable(), ggtheme = theme_bw())
Log-Rank test
survdiff(Surv(CMTAR, CM) ~ Tx,data=SDCTCOW,rho=0)Call:
survdiff(formula = Surv(CMTAR, CM) ~ Tx, data = SDCTCOW, rho = 0)
N Observed Expected (O-E)^2/E (O-E)^2/V
Tx=Blanket 407 59 52 0.954 1.428
Tx=Culture 410 50 54 0.298 0.454
Tx=Algorithm 394 48 51 0.180 0.267
Chisq= 1.4 on 2 degrees of freedom, p= 0.5 Cox proportional hazards regression for clinical mastitis (1-120 DIM)
Model building plan
Model type: Cox proportional hazards analysis with farm included as a cluster variable (robust sandwich standard error estimator) to account for lack of indepedence.
Step 1: Identify potential confouders using a directed acyclic graph (DAG)
Step 2: Identify correlated variables using pearson and kendalls correlation coefficients
Step 3: Create model with all potential confounders
Step 4: Investigate if covariates meet proportional hazards assumption
Step 5: Investigate potential effect measure modification
Step 6: Remove unneccesary covariates in backwards stepwise fashion using 10% rule (i.e. if hazard ratio for algorithm or culture changes by >10% after removing the covariate, the covariate is retained in the model)
Step 7: Report final model
Step 1: DAG for clincal mastitis
This is used to identify variables that could be confounders if they are not balanced between treatment groups.
library(DiagrammeR)
mermaid("graph LR
T(Treatment)-->U(Clinical mastitis)
A(Age)-->T
P(Parity)-->T
M(Yield at dry-off)-->T
S(SCC during prev lactation)-->T
C(CM in prev lact)-->T
A-->U
P-->U
M-->U
S-->U
C-->U
C-->M
P-->C
P-->S
P-->M
A-->P
A-->C
A-->S
A-->M
M-->S
C-->S
style A fill:#FFFFFF, stroke-width:0px
style T fill:#FFFFFF, stroke-width:2px
style P fill:#FFFFFF, stroke-width:0px
style M fill:#FFFFFF, stroke-width:0px
style S fill:#FFFFFF, stroke-width:0px
style C fill:#FFFFFF, stroke-width:0px
style I fill:#FFFFFF, stroke-width:0px
style U fill:#FFFFFF, stroke-width:2px
")According to this DAG, I may need to control for the following variables.
Parity [“Parity”]
Age [“Age”]
Yield at most recent test before dry off [“DOMY”]
Somatic cell count at last herd test during previous lactation [“DOSCC” or “PrevSCCHi”] <- likely to be correlated
Clinical mastitis in previous lactation [“PrevCM”]
Step 3: Create model with all potential covariates
Cox proportional hazards regression for clinical mastitis (1-120 DIM)
Cows that did not calve or that had long/short dry periods are excluded. Cows with clinical mastitis prior to calving are included.
Reasons for R censor = 120 DIM or culling/death
library(broom)
library(survival)
SR <- coxph(Surv(CMTAR, CM) ~ Parity + DOMY + DOSCC + PrevCM + Tx + cluster(FARMID), data=SDCTCOW)
coxph(Surv(CMTAR, CM) ~ Parity + DOMY + DOSCC + PrevCM + Tx + cluster(FARMID), data=SDCTCOW)Call:
coxph(formula = Surv(CMTAR, CM) ~ Parity + DOMY + DOSCC + PrevCM +
Tx + cluster(FARMID), data = SDCTCOW)
coef exp(coef) se(coef) robust se z p
Parity2 0.262972 1.300790 0.198272 0.127747 2.059 0.0395
Parity3 0.328493 1.388873 0.204918 0.292867 1.122 0.2620
DOMY -0.006115 0.993904 0.010031 0.018794 -0.325 0.7449
DOSCC 0.028361 1.028767 0.076123 0.105980 0.268 0.7890
PrevCM1 0.357508 1.429761 0.212265 0.228486 1.565 0.1177
TxCulture -0.193527 0.824048 0.192911 0.177916 -1.088 0.2767
TxAlgorithm -0.173991 0.840304 0.194505 0.162762 -1.069 0.2851
Likelihood ratio test=10.21 on 7 df, p=0.1771
n= 1211, number of events= 157 #tidy(coxph(Surv(CMTAR, CM) ~ Tx + cluster(FARMID), data=SDCTCOW))library(broom)
library(survival)
SR <- coxph(Surv(CMTAR, CM) ~ Parity + DOMY + DOSCC + PrevCM + Tx + cluster(FARMID), data=SDCTCOW)
coxph(Surv(CMTAR, CM) ~ Parity + DOMY + DOSCC + PrevCM + Tx + cluster(FARMID), data=SDCTCOW)Call:
coxph(formula = Surv(CMTAR, CM) ~ Parity + DOMY + DOSCC + PrevCM +
Tx + cluster(FARMID), data = SDCTCOW)
coef exp(coef) se(coef) robust se z p
Parity2 0.262972 1.300790 0.198272 0.127747 2.059 0.0395
Parity3 0.328493 1.388873 0.204918 0.292867 1.122 0.2620
DOMY -0.006115 0.993904 0.010031 0.018794 -0.325 0.7449
DOSCC 0.028361 1.028767 0.076123 0.105980 0.268 0.7890
PrevCM1 0.357508 1.429761 0.212265 0.228486 1.565 0.1177
TxCulture -0.193527 0.824048 0.192911 0.177916 -1.088 0.2767
TxAlgorithm -0.173991 0.840304 0.194505 0.162762 -1.069 0.2851
Likelihood ratio test=10.21 on 7 df, p=0.1771
n= 1211, number of events= 157 #tidy(coxph(Surv(CMTAR, CM) ~ Tx + cluster(FARMID), data=SDCTCOW))Step 4: Schoenfield tests to assess assumption of proportional hazards for explanatory variables.
Parity, Tx and DOSCC violate the assumption of proportional hazards
SR <- cox.zph(SR)
SR rho chisq p
Parity2 -0.1172 4.313 0.037831
Parity3 -0.1245 11.662 0.000638
DOMY 0.0370 1.843 0.174616
DOSCC 0.1043 5.489 0.019134
PrevCM1 0.0207 0.394 0.530189
TxCulture 0.1564 9.946 0.001612
TxAlgorithm 0.0207 0.480 0.488505
GLOBAL NA 14.007 0.051053
Schoenfield residual plots to assess assumption of proportional hazards.
ggcoxzph(SR,var = c("TxCulture","TxAlgorithm"))
ggcoxzph(SR,var = c("DOSCC", "DOMY"))
ggcoxzph(SR,var = c("Parity2","Parity3"))
ggcoxzph(SR,var = c("PrevCM1"))
I will deal with these covariates using stratification (i.e. fitting seperate baseline hazards)
I must create categorical variables to allow for this (not shown here).
Dry off milk yield [“DOMY”] -> median split (“DOMYcat” = 0 / 1).
Dry off SCC - split at 200,000 cells (subclinical mastitis [“DOSCM”] = 0 / 1)
Show new variables
SDCTCOWcheck <- SDCTCOW %>% select(Tx,CM,CMTAR,Cull2,Cull2TAR,DOSCC,DOSCM,DOMY,DOMYcat,Parity,PrevCM)
head(SDCTCOWcheck)
Run new cox model with stratified variables
SR <- coxph(Surv(CMTAR, CM) ~ strata(Parity) + strata(DOMYcat) + strata(DOSCM) + PrevCM + Tx + cluster(FARMID), data=SDCTCOW)
tidy(coxph(Surv(CMTAR, CM) ~ strata(Parity) + strata(DOMYcat) + strata(DOSCM) + PrevCM + Tx + cluster(FARMID), data=SDCTCOW))
Repeat Schoenfeld residuals
Tx no longer p < 0.05, but is close to the threshold (p = 0.08).
SR <- cox.zph(SR)
SR rho chisq p
PrevCM1 0.0273 0.185 0.6674
TxCulture 0.1393 3.094 0.0786
TxAlgorithm 0.0442 0.419 0.5173
GLOBAL NA 3.320 0.3449
Will split time into 1-60 and 61-120 DIM to see if this difference is meaningful.
Cox model 1-60 DIM
Beta coefficient for Culture = -0.43, Algorithm = -0.08
SR <- coxph(Surv(CMTAR60, CM60) ~ strata(Parity) + strata(DOMYcat) + strata(DOSCM) + PrevCM + Tx + cluster(FARMID), data=SDCTCOW)
tidy(coxph(Surv(CMTAR60, CM60) ~ strata(Parity) + strata(DOMYcat) + strata(DOSCM) + PrevCM + Tx + cluster(FARMID), data=SDCTCOW))
Cox model 61-120 DIM Beta coefficient for Culture = 0.008, Algorithm = -.401
SR <- coxph(Surv(CMTAR60120, CM60120) ~ strata(Parity) + strata(DOMYcat) + strata(DOSCM) + PrevCM + Tx + cluster(FARMID), data=SDCTCOW)
tidy(coxph(Surv(CMTAR60120, CM60120) ~ strata(Parity) + strata(DOMYcat) + strata(DOSCM) + PrevCM + Tx + cluster(FARMID), data=SDCTCOW))Decision: These differences aren’t sufficient in my opinion to warrant pursuing the 1-60, 61-120 DIM models. An average of the two is appropriate.
Step 5: Investigate effect measure modification
Unable to evluate interaction between Tx & FARMID using Cox regression - Bizzare SE’s, won’t converge.
mm0 <- coxph(Surv(CMTAR, CM) ~ Tx + FARMID + Tx:FARMID + PrevCM + strata(Parity) + strata(DOMYcat) + strata(DOSCM), data=SDCTCOW)Loglik converged before variable 19 ; coefficient may be infinite. tidy(mm0)
Will use logistic regression to assess for interaction between Tx and farm: P > 0.05.
mm0 <- glm(CM ~ Tx*FARMID + PrevCM + Parity + DOMYcat + DOSCM, data=SDCTCOW, family=binomial)
car::Anova(mm0)Analysis of Deviance Table (Type II tests)
Response: CM
LR Chisq Df Pr(>Chisq)
Tx 0.881 2 0.6437
FARMID 35.339 6 3.704e-06 ***
PrevCM 0.763 1 0.3824
Parity 1.923 2 0.3823
DOMYcat 1.466 1 0.2260
DOSCM 0.757 1 0.3843
Tx:FARMID 12.592 12 0.3994
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1Decision: No treatment by herd effect measure modification
Assess effect measure modification with previous clinical mastitis using Cox regression.
a <- coxph(Surv(CMTAR, CM) ~ factor(Tx)*factor(PrevCM) + strata(Parity) + cluster(FARMID) + strata(DOMYcat) + strata(DOSCM), data=SDCTCOW)
library(aod)
wald.test(b = coef(a), Sigma = vcov(a), Terms = 4:5)Wald test:
----------
Chi-squared test:
X2 = 11.0, df = 2, P(> X2) = 0.004Wald test for interaction is p < 0.05.
Model output with interaction
mm0 <- coxph(Surv(CMTAR, CM) ~ Tx*PrevCM + strata(Parity) + strata(DOMYcat) + strata(DOSCM) + cluster(FARMID), data=SDCTCOW)
tidy(mm0)According to this interaction model, the HR (referent = Blanket Tx, no prev CM) for each group are:
Blanket:Prev0 = 1
Blanket:Prev1 = 2.1
Culture:PrevCM0 = 0.93
Culture:PrevCM1 = 0.97
Algorithm:PrevCM0 = 0.93
Algorithm:PrevCM1 = 1.14
This is a counter-intuitive result: hazards of clinical mastitis are higher in blanket cows with previous history of CM.
However, these effect estimates have wide confidence intervals, and this interaction model adds unnecessary complexity and does not change our conclusions.
Decision: Report main effects.
Step 6: Remove unnecessary covariates (backwards selection using 10% rule)
The order for removing covariates will be in increasing likelihood of being a confounder, which is based on my knowledge about the variables and their distribution in treatment groups.
This order will be: DOMYcat, DOSCM, Parity, PrevCM
Step 6a: Full model
mm0 <- tidy(coxph(Surv(CMTAR, CM) ~ strata(Parity) + strata(DOMYcat) + strata(DOSCM) + PrevCM + Tx + cluster(FARMID), data=SDCTCOW)) %>% select("term","estimate")
mm0$HR <- exp(mm0$estimate)
mm0
Step 6b: removing DOMYcat
mm0 <- tidy(coxph(Surv(CMTAR, CM) ~ strata(Parity) + strata(DOSCM) + PrevCM + Tx + cluster(FARMID), data=SDCTCOW))%>% select("term","estimate")
mm0$HR <- exp(mm0$estimate)
mm0Both HR (Culture and Algorithm) changed by <10%. DOMYcat stays out.
Step 6c: removing DOSCM
mm0 <- tidy(coxph(Surv(CMTAR, CM) ~ strata(Parity) + PrevCM + Tx + cluster(FARMID), data=SDCTCOW)) %>% select("term","estimate")
mm0$HR <- exp(mm0$estimate)
mm0Both HR changed by <10%. DOMYcat stays out.
Step 6d: removing Parity
mm0 <- tidy(coxph(Surv(CMTAR, CM) ~ PrevCM + Tx + cluster(FARMID), data=SDCTCOW)) %>% select("term","estimate")
mm0$HR <- exp(mm0$estimate)
mm0Both HR changed by <10%. Parity stays out.
Step 6e: removing PrevCM
mm0 <- tidy(coxph(Surv(CMTAR, CM) ~ Tx + cluster(FARMID), data=SDCTCOW)) %>% select("term","estimate")
mm0$HR <- exp(mm0$estimate)
mm0Both HR changed by <10%. PrevCM stays out.
Step 7a: Reporting final cox model for clinical mastitis (1-120 DIM)
No covariates included, as no evidence for confounding.
CoxCM <- tidy(coxph(Surv(CMTAR, CM) ~ Tx + cluster(FARMID), data=SDCTCOW))
CoxCM$HR <- exp(CoxCM$estimate)
CoxCM$LCL <- exp(CoxCM$conf.low)
CoxCM$UCL <- exp(CoxCM$conf.high)
CoxCM <- CoxCM %>% select(term,HR,LCL,UCL,robust.se,p.value)
CoxCMChecking Schoenfield residuals for Tx in final model
SR <- coxph(Surv(CMTAR, CM) ~ Tx + cluster(FARMID), data=SDCTCOW)
cox.zph(SR) rho chisq p
TxCulture 0.12471 2.077891 0.149
TxAlgorithm -0.00225 0.000429 0.983
GLOBAL NA 2.596282 0.273SR <- cox.zph(SR)
ggcoxzph(SR,var = c("TxCulture","TxAlgorithm"))
Sufficient evidence to use Cox model.
Step 7b: Reporting final cox model for clinical mastitis (1-120 DIM) using P-value based backwards stepwise selection
CoxCM <- coxph(Surv(CMTAR, CM) ~ Tx + Parity + cluster(FARMID), data=SDCTCOW) %>% tidy()
CoxCM$HR <- exp(CoxCM$estimate)
CoxCM$LCL <- exp(CoxCM$conf.low)
CoxCM$UCL <- exp(CoxCM$conf.high)
CoxCM <- CoxCM %>% select(term,HR,LCL,UCL,robust.se,p.value)
CoxCMEstimates are very similar to those found using 10% rule.
Outcome 2: Culling / Death
Kaplan Meier curve & log rank test for culling/death
Kaplan Meier curve
KM <- survfit(Surv(Cull2TAR, Cull2) ~ Tx, data = SDCTCOW)
ggsurvplot(KM, data = SDCTCOW, title = "", pval = T, conf.int = F,risk.table.col = "Tx",risk.table = T, risk.table.y.text.col = TRUE , surv.plot.height = 5, legend.labs = c("Blanket","Culture","Algorithm"), tables.theme = theme_cleantable(), ggtheme = theme_bw())
Log-Rank test
survdiff(Surv(Cull2TAR, Cull2) ~ Tx,data=SDCTCOW,rho=0)Call:
survdiff(formula = Surv(Cull2TAR, Cull2) ~ Tx, data = SDCTCOW,
rho = 0)
N Observed Expected (O-E)^2/E (O-E)^2/V
Tx=Blanket 407 44 42.1 0.0891 0.134
Tx=Culture 410 40 43.0 0.2128 0.323
Tx=Algorithm 394 42 40.9 0.0290 0.043
Chisq= 0.3 on 2 degrees of freedom, p= 0.8 Cox proportional hazards regression for culling or death (1-120 DIM)
Model building plan
Model type: Cox proportional hazards analysis with farm included as a cluster variable to account for lack of indepedence.
Step 1: Identify potential confouders using a directed acyclic graph (DAG)
Step 2: Identify correlated variables using pearson and kendalls correlation coefficients
Step 3: Create model with all potential confounders
Step 4: Investigate if covariates meet proportional hazards assumption
Step 5: Investigate potential effect measure modification
Step 6: Remove unneccesary covariates in backwards stepwise fashion using 10% rule (i.e. if HR changes by >10% after removal of covariate, the covariate is retained in the model)
Step 7: Report final model
Steps 1 & 2: Identify potential confouders using a directed acyclic graph (DAG)
library(DiagrammeR)
mermaid("graph LR
T(Treatment)-->U(Culling or Death)
A(Age)-->T
P(Parity)-->T
M(Yield at dry-off)-->T
S(SCC during prev lactation)-->T
C(CM in prev lact)-->T
A-->U
P-->U
M-->U
S-->U
C-->U
C-->M
P-->C
P-->S
P-->M
A-->P
A-->C
A-->S
A-->M
M-->S
C-->S
style A fill:#FFFFFF, stroke-width:0px
style T fill:#FFFFFF, stroke-width:2px
style P fill:#FFFFFF, stroke-width:0px
style M fill:#FFFFFF, stroke-width:0px
style S fill:#FFFFFF, stroke-width:0px
style C fill:#FFFFFF, stroke-width:0px
style I fill:#FFFFFF, stroke-width:0px
style U fill:#FFFFFF, stroke-width:2px
")According to this DAG, I may need to control for the following variables:
Parity [“Parity”]. Will not offer Age [“Age”] as it is highly correlated.
Yield at most recent test before dry off [“DOMY”]
Somatic cell count at last herd test during previous lactation [“DOSCC”] <- will not offer PrevSCCHI as it was correlated with DOSCC
Clinical mastitis in previous lactation [“PrevCM”]
Step 3: Create model with all potential confounders
Cows that did not calve are excluded (left censored)
Reason for R censor = 120 DIM
library(broom)
library(survival)
coxph(Surv(Cull2TAR, Cull2) ~ Parity + DOMY + DOSCC + PrevCM + Tx + cluster(FARMID), data=SDCTCOW)Call:
coxph(formula = Surv(Cull2TAR, Cull2) ~ Parity + DOMY + DOSCC +
PrevCM + Tx + cluster(FARMID), data = SDCTCOW)
coef exp(coef) se(coef) robust se z p
Parity2 0.39524 1.48475 0.25866 0.21257 1.859 0.06298
Parity3 1.13923 3.12436 0.23351 0.23115 4.929 8.29e-07
DOMY -0.01446 0.98564 0.01107 0.01471 -0.983 0.32560
DOSCC 0.08981 1.09396 0.08514 0.05241 1.714 0.08660
PrevCM1 0.26788 1.30719 0.22819 0.09322 2.874 0.00406
TxCulture -0.09850 0.90619 0.21940 0.21233 -0.464 0.64271
TxAlgorithm 0.01074 1.01079 0.21583 0.23312 0.046 0.96327
Likelihood ratio test=44.51 on 7 df, p=1.706e-07
n= 1211, number of events= 126
Step 4: Investigate if covariates meet proportional hazards assumption
Both DOSCC and DOMY don’t meet the assumption of proportional hazards
SR <- cox.zph(coxph(Surv(Cull2TAR, Cull2) ~ Parity + DOMY + DOSCC + PrevCM + Tx + cluster(FARMID), data=SDCTCOW))
SR rho chisq p
Parity2 -0.1157 2.607 0.10639
Parity3 -0.0343 0.209 0.64753
DOMY 0.1521 7.402 0.00652
DOSCC 0.1462 5.471 0.01934
PrevCM1 -0.1220 2.530 0.11172
TxCulture 0.1222 3.313 0.06872
TxAlgorithm 0.0815 1.396 0.23741
GLOBAL NA 8.182 0.31686Schoenfield residual plots
ggcoxzph(SR,var = c("TxCulture","TxAlgorithm"))
DOSCC and DOMY don’t meet the assumption of proportional hazards
ggcoxzph(SR,var = c("DOSCC", "DOMY"))
ggcoxzph(SR,var = c("Parity2","Parity3"))
ggcoxzph(SR,var = c("PrevCM1"))
Refit cox model with seperate baselines (stratified) for DOMY and DOSCC.
No evidence of other time-varying covariates or predictors (Tx).
SR <- coxph(Surv(Cull2TAR, Cull2) ~ Parity + strata(DOMYcat) + strata(DOSCM) + PrevCM + Tx + cluster(FARMID), data=SDCTCOW)
SR <- cox.zph(SR)
SR rho chisq p
Parity2 -0.05884 0.412749 0.521
Parity3 -0.00113 0.000217 0.988
PrevCM1 -0.06962 0.500950 0.479
TxCulture 0.10086 1.689182 0.194
TxAlgorithm 0.07741 1.091294 0.296
GLOBAL NA 2.214228 0.819
Step 5: Assess for effect measure modification
Tx : FARMID - could not assess this using Cox regression as model did not converge or reported unusually large SEs for some interaction terms.
SR <- coxph(Surv(Cull2TAR, Cull2) ~ Tx*FARMID + Parity + PrevCM + Parity + strata(DOMYcat) + strata(DOSCM), data=SDCTCOW)Loglik converged before variable 20 ; coefficient may be infinite. tidy(SR)
I will assess effect Tx:farm interaction using logistic regression:
Wald test for interaction term was P > 0.05.
mm0 <- glm(CM ~ Tx*FARMID + Parity + PrevCM + Parity + DOMYcat + DOSCM, data=SDCTCOW)
car::Anova(mm0)Analysis of Deviance Table (Type II tests)
Response: CM
LR Chisq Df Pr(>Chisq)
Tx 0.924 2 0.6300
FARMID 33.573 6 8.132e-06 ***
Parity 1.953 2 0.3767
PrevCM 1.039 1 0.3082
DOMYcat 1.573 1 0.2097
DOSCM 0.810 1 0.3683
Tx:FARMID 10.174 12 0.6007
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
Assess for other interactions using Wald testfrom Cox regression.
Wald test was P > 0.05 for Tx:PrevCM interaction .
mm0 <- coxph(Surv(Cull2TAR, Cull2) ~ Tx*PrevCM + Parity + strata(DOMYcat) + strata(DOSCM) + cluster(FARMID), data=SDCTCOW)
summary(mm0)Call:
coxph(formula = Surv(Cull2TAR, Cull2) ~ Tx * PrevCM + Parity +
strata(DOMYcat) + strata(DOSCM) + cluster(FARMID), data = SDCTCOW)
n= 1211, number of events= 126
coef exp(coef) se(coef) robust se z Pr(>|z|)
TxCulture -0.02411 0.97618 0.24667 0.23595 -0.102 0.9186
TxAlgorithm 0.05495 1.05649 0.24266 0.27674 0.199 0.8426
PrevCM1 0.49435 1.63943 0.36416 0.20400 2.423 0.0154 *
Parity2 0.41575 1.51550 0.25731 0.21398 1.943 0.0520 .
Parity3 1.17829 3.24881 0.23075 0.25675 4.589 4.45e-06 ***
TxCulture:PrevCM1 -0.40970 0.66385 0.53890 0.33877 -1.209 0.2265
TxAlgorithm:PrevCM1 -0.21937 0.80303 0.53502 0.70719 -0.310 0.7564
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
exp(coef) exp(-coef) lower .95 upper .95
TxCulture 0.9762 1.0244 0.6147 1.550
TxAlgorithm 1.0565 0.9465 0.6142 1.817
PrevCM1 1.6394 0.6100 1.0991 2.445
Parity2 1.5155 0.6598 0.9964 2.305
Parity3 3.2488 0.3078 1.9642 5.374
TxCulture:PrevCM1 0.6638 1.5064 0.3417 1.290
TxAlgorithm:PrevCM1 0.8030 1.2453 0.2008 3.211
Concordance= 0.64 (se = 0.03 )
Likelihood ratio test= 33.79 on 7 df, p=2e-05
Wald test = 1281 on 7 df, p=<2e-16
Score (logrank) test = 36.55 on 7 df, p=6e-06, Robust = 7 p=0.4
(Note: the likelihood ratio and score tests assume independence of
observations within a cluster, the Wald and robust score tests do not).wald.test(b = coef(mm0), Sigma = vcov(mm0), Terms = 6:7)Wald test:
----------
Chi-squared test:
X2 = 1.6, df = 2, P(> X2) = 0.46
Wald test was P < 0.05 for Tx:Parity interaction .
mm0 <- coxph(Surv(Cull2TAR, Cull2) ~ Tx*Parity + PrevCM + strata(DOMYcat) + strata(DOSCM) + cluster(FARMID), data=SDCTCOW)
summary(mm0)Call:
coxph(formula = Surv(Cull2TAR, Cull2) ~ Tx * Parity + PrevCM +
strata(DOMYcat) + strata(DOSCM) + cluster(FARMID), data = SDCTCOW)
n= 1211, number of events= 126
coef exp(coef) se(coef) robust se z Pr(>|z|)
TxCulture 0.5382 1.7130 0.5005 0.4124 1.305 0.191869
TxAlgorithm 0.5758 1.7786 0.5080 0.5537 1.040 0.298327
Parity2 0.8960 2.4498 0.4961 0.4546 1.971 0.048749 *
Parity3 1.7523 5.7680 0.4572 0.4022 4.357 1.32e-05 ***
PrevCM1 0.3018 1.3522 0.2277 0.1047 2.882 0.003946 **
TxCulture:Parity2 -0.6590 0.5173 0.6640 0.7136 -0.924 0.355691
TxAlgorithm:Parity2 -0.6252 0.5352 0.6531 0.6359 -0.983 0.325518
TxCulture:Parity3 -0.8959 0.4082 0.5861 0.2552 -3.511 0.000446 ***
TxAlgorithm:Parity3 -0.7392 0.4775 0.5900 0.4252 -1.739 0.082111 .
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
exp(coef) exp(-coef) lower .95 upper .95
TxCulture 1.7130 0.5838 0.7633 3.8440
TxAlgorithm 1.7786 0.5622 0.6009 5.2648
Parity2 2.4498 0.4082 1.0049 5.9721
Parity3 5.7680 0.1734 2.6224 12.6871
PrevCM1 1.3522 0.7395 1.1014 1.6602
TxCulture:Parity2 0.5173 1.9329 0.1278 2.0949
TxAlgorithm:Parity2 0.5352 1.8686 0.1539 1.8610
TxCulture:Parity3 0.4082 2.4496 0.2476 0.6731
TxAlgorithm:Parity3 0.4775 2.0943 0.2075 1.0987
Concordance= 0.651 (se = 0.025 )
Likelihood ratio test= 35.95 on 9 df, p=4e-05
Wald test = 89593 on 9 df, p=<2e-16
Score (logrank) test = 38.57 on 9 df, p=1e-05, Robust = 7 p=0.6
(Note: the likelihood ratio and score tests assume independence of
observations within a cluster, the Wald and robust score tests do not).wald.test(b = coef(mm0), Sigma = vcov(mm0), Terms = 6:9)Wald test:
----------
Chi-squared test:
X2 = 658.3, df = 4, P(> X2) = 0.0
Investigate potential effect measure modification by doing stratified models
Parity = 1 Frequency table
table(SDCTCOW$Tx[SDCTCOW$Parity==1],SDCTCOW$Cull2[SDCTCOW$Parity==1])
0 1
Blanket 152 6
Culture 173 12
Algorithm 156 11
Model (Parity = 1)
mm0 <- coxph(Surv(Cull2TAR, Cull2) ~ Tx + PrevCM + strata(DOMYcat) + strata(DOSCM) + cluster(FARMID), data=SDCTCOW[SDCTCOW$Parity==1,]) %>% tidy()
mm0$HR <- exp(mm0$estimate)
mm0$LCL <- exp(mm0$conf.low)
mm0$UCL <- exp(mm0$conf.high)
mm0 %>% select("term","HR","LCL","UCL")
Parity = 2
Frequency table
table(SDCTCOW$Tx[SDCTCOW$Parity==2],SDCTCOW$Cull2[SDCTCOW$Parity==2])
0 1
Blanket 120 13
Culture 99 9
Algorithm 112 11mm0 <- coxph(Surv(Cull2TAR, Cull2) ~ Tx + PrevCM + strata(DOMYcat) + strata(DOSCM) + cluster(FARMID), data=SDCTCOW[SDCTCOW$Parity==2,]) %>% tidy()
mm0$HR <- exp(mm0$estimate)
mm0$LCL <- exp(mm0$conf.low)
mm0$UCL <- exp(mm0$conf.high)
mm0 %>% select("term","HR","LCL","UCL")
Parity = 3 or greater
Frequency table
table(SDCTCOW$Tx[SDCTCOW$Parity==3],SDCTCOW$Cull2[SDCTCOW$Parity==3])
0 1
Blanket 91 25
Culture 98 19
Algorithm 84 20mm0 <- coxph(Surv(Cull2TAR, Cull2) ~ Tx + PrevCM + strata(DOMYcat) + strata(DOSCM) + cluster(FARMID), data=SDCTCOW[SDCTCOW$Parity==3,]) %>% tidy
mm0$HR <- exp(mm0$estimate)
mm0$LCL <- exp(mm0$conf.low)
mm0$UCL <- exp(mm0$conf.high)
mm0 %>% select("term","HR","LCL","UCL")This finding suggests that potential culling risks associated with SDCT may be higher in parity 1 cows than in multiparous animals. However these HR estimates in all 3 parity groups are imprecise (wide confidence intervals) because of a small number of culling/death events. For example, in lactation 1 animals: 6 in BDCT, 11 in Algorithm and 12 in Culture. Therefore, this difference in effects across levels of parity may be due to chance alone, and not worth investigating too closely.
Decision: Report main effects.
Step 6: Remove unnecessary covariates (backwards selection using 10% rule)
The order for removing covariates will be in increasing likelihood of being a confounder, which is based on my knowledge about the variables and their distribution in treatment groups.
This order will be: DOMYcat, DOSCM, PrevCM, Parity
Step 6a: Full model
mm0 <- coxph(Surv(Cull2TAR, Cull2) ~ Tx + PrevCM + Parity + strata(DOMYcat) + strata(DOSCM) + cluster(FARMID), data=SDCTCOW) %>% tidy() %>% select("term","estimate")
mm0$HR <- exp(mm0$estimate)
mm0
Step 6b: removing DOMYcat
mm0 <- coxph(Surv(Cull2TAR, Cull2) ~ Tx + PrevCM + Parity + strata(DOSCM) + cluster(FARMID), data=SDCTCOW) %>% tidy() %>% select("term","estimate")
mm0$HR <- exp(mm0$estimate)
mm0HR for culture and algorithm didn’t change by >10%. DOMYcat stays out.
Step 6c: removing DOSCM
mm0 <- coxph(Surv(Cull2TAR, Cull2) ~ Tx + PrevCM + Parity + cluster(FARMID), data=SDCTCOW) %>% tidy() %>% select("term","estimate")
mm0$HR <- exp(mm0$estimate)
mm0HR changed by <10%. DOSCM stays out.
Step 6d: removing PrevCM
mm0 <- coxph(Surv(Cull2TAR, Cull2) ~ Tx + Parity + cluster(FARMID), data=SDCTCOW) %>% tidy() %>% select("term","estimate")
mm0$HR <- exp(mm0$estimate)
mm0HR changed by <10%. Leave PrevCM out.
Step 6e: removing Parity
mm0 <- coxph(Surv(Cull2TAR, Cull2) ~ Tx + cluster(FARMID), data=SDCTCOW) %>% tidy() %>% select("term","estimate")
mm0$HR <- exp(mm0$estimate)
mm0HR changed by <10%. Parity stays out.
Step 7a: Final cox model for culling or death (1 - 120 DIM)
No covariates included, as no evidence for confounding.
CoxCull <- coxph(Surv(Cull2TAR, Cull2) ~ Tx + cluster(FARMID), data=SDCTCOW) %>% tidy()
CoxCull$HR <- exp(CoxCull$estimate)
CoxCull$LCL <- exp(CoxCull$conf.low)
CoxCull$UCL <- exp(CoxCull$conf.high)
CoxCull <- CoxCull %>% select(term,HR,LCL,UCL,robust.se,p.value)
CoxCullCheck schoenfield residuals for final model
SR <- coxph(Surv(Cull2TAR, Cull2) ~ Tx + cluster(FARMID), data=SDCTCOW)
cox.zph(SR) rho chisq p
TxCulture 0.1100 1.52 0.217
TxAlgorithm 0.0951 1.12 0.290
GLOBAL NA 1.53 0.466SR <- cox.zph(SR)
ggcoxzph(SR,var = c("TxCulture","TxAlgorithm"))
Step 7b: Final cox model for culling or death (1 - 120 DIM) using P-value based backwards stepwise selection
CoxCull <- coxph(Surv(CullTAR, Cull2) ~ Tx + Parity + DOSCC + PrevCM + cluster(FARMID), data=SDCTCOW) %>% tidy()
CoxCull$HR <- exp(CoxCull$estimate)
CoxCull$LCL <- exp(CoxCull$conf.low)
CoxCull$UCL <- exp(CoxCull$conf.high)
CoxCull <- CoxCull %>% select(term,HR,LCL,UCL,robust.se,p.value)
CoxCullEstimates are similar to 10% rule based approach
Changing datasets: DHI dataset with multiple rows per cow
Dataset example
SDCTCOWDHI$Tx <-
factor(SDCTCOWDHI$Tx,
levels=c(0,1,2),
labels=c("Blanket",
"Culture",
"Algorithm"))
DHIcheck <- SDCTCOWDHI %>% select(Tx,CowID,FARMID,DIM,TestDIMcat20,LSCC,MY,Parity,PrevCM,DOMY,DOSCC)
head(DHIcheck, n=10)
Inspect data
DHIoutcome <- SDCTCOWDHI %>% select(SCC,MY,SCM)
DHIoutcome$logSCC <- log(DHIoutcome$SCC + 1)
#summarytools::dfSummary(DHIoutcome, style='grid')
print(summarytools::dfSummary(DHIoutcome, valid.col=FALSE, graph.magnif=0.8, style="grid"), method = "render")
Data Frame Summary
DHIoutcome
Dimensions: 3992 x 4Duplicates: 967
| No | Variable | Stats / Values | Freqs (% of Valid) | Graph | Missing | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | SCC [numeric] | Mean (sd) : 184.4 (678.2) min < med < max: 0 < 39 < 9999 IQR (CV) : 89 (3.7) | 608 distinct values |  |
3 (0.08%) | ||||||||||||
| 2 | MY [numeric] | Mean (sd) : 49.1 (10.5) min < med < max: 0.9 < 49.4 < 93.4 IQR (CV) : 13.2 (0.2) | 149 distinct values |  |
2 (0.05%) | ||||||||||||
| 3 | SCM [numeric] | Min : 0 Mean : 0.1 Max : 1 |
|
 |
3 (0.08%) | ||||||||||||
| 4 | logSCC [numeric] | Mean (sd) : 3.9 (1.4) min < med < max: 0 < 3.7 < 9.2 IQR (CV) : 1.7 (0.3) | 608 distinct values |  |
3 (0.08%) |
Generated by summarytools 0.9.3 (R version 3.6.0)
2019-07-02
NAVery little missing data. Will do complete case analysis.
Assessing normality for continuous outcome variables
Somatic cell count (log x 10,000 cells)
qqPlot(log(SDCTCOWDHI$SCC+1))
hist(log(SDCTCOWDHI$SCC+1))
Milk yield (kg)
qqPlot(SDCTCOWDHI$MY)
hist(SDCTCOWDHI$MY)
Outcome 3: Somatic cell count
Modelling plan
Model type: Linear mixed models, random intercepts for farm and cow will be fitted to account for repeated measures within cows, and clustering of cows within herds
Step 1: Identify potential confouders using a directed acyclic graph (DAG)
Step 2: Identify correlated variables using pearson and kendalls correlation coefficients
Step 3: Create model with all potential confounders
Step 4: Investigate potential effect measure modification
Step 5: Remove unneccesary covariates in backwards stepwise fashion using 10% rule (i.e. if estimated difference in SCC changes by >10% after removing the covariate, the covariate is retained in the model)
Step 6: Report final model
Step 7: Model diagnostics
Step 1 & 2 Identify potential confounders using a DAG
library(DiagrammeR)
mermaid("graph LR
T(Treatment)-->U(SCC)
A(Age)-->T
P(Parity)-->T
M(Yield at dry-off)-->T
S(SCC during prev lactation)-->T
C(CM in prev lact)-->T
D(Days in milk at test)-->U
D-->T
A-->U
P-->U
M-->U
S-->U
C-->U
C-->M
P-->C
P-->S
P-->M
A-->P
A-->C
A-->S
A-->M
M-->S
C-->S
style A fill:#FFFFFF, stroke-width:0px
style T fill:#FFFFFF, stroke-width:2px
style P fill:#FFFFFF, stroke-width:0px
style M fill:#FFFFFF, stroke-width:0px
style S fill:#FFFFFF, stroke-width:0px
style C fill:#FFFFFF, stroke-width:0px
style I fill:#FFFFFF, stroke-width:0px
style D fill:#FFFFFF, stroke-width:0px
style U fill:#FFFFFF, stroke-width:2px
")According to this DAG, I may need to control for the following variables:
Parity [“Parity”] <- Age not offered as highly correlated
Yield at most recent test before dry off [“DOMY”]
Somatic cell count at last herd test during previous lactation [“DOSCC”] <- PrevSCCHI not offered as correlated
Clinical mastitis in previous lactation [“PrevCM”]
Days in milk at herd test (category, 0-20 “10”, 21-40 “30” etc) [“TestDIMcat20”]
Step 3: Create model with all potential confounders
mm0 <- lmer(LSCC ~ Tx + TestDIMcat20 + Parity + DOSCC + DOMY + PrevCM + (1|FARMID/CowID), data=SDCTCOWDHI)
summary(mm0)Linear mixed model fit by REML ['lmerMod']
Formula: LSCC ~ Tx + TestDIMcat20 + Parity + DOSCC + DOMY + PrevCM + (1 | FARMID/CowID)
Data: SDCTCOWDHI
REML criterion at convergence: 13110.3
Scaled residuals:
Min 1Q Median 3Q Max
-5.5975 -0.4985 -0.1040 0.4340 3.9735
Random effects:
Groups Name Variance Std.Dev.
CowID:FARMID (Intercept) 0.61791 0.7861
FARMID (Intercept) 0.02027 0.1424
Residual 1.14078 1.0681
Number of obs: 3989, groups: CowID:FARMID, 1178; FARMID, 7
Fixed effects:
Estimate Std. Error t value
(Intercept) 3.338254 0.204372 16.334
TxCulture 0.047878 0.070138 0.683
TxAlgorithm 0.074745 0.070822 1.055
TestDIMcat2030 -0.576961 0.063393 -9.101
TestDIMcat2050 -0.557223 0.060950 -9.142
TestDIMcat2070 -0.394721 0.059969 -6.582
TestDIMcat2090 -0.272158 0.065089 -4.181
TestDIMcat20110 -0.084757 0.060209 -1.408
Parity2 0.110743 0.070611 1.568
Parity3 0.259957 0.074986 3.467
DOSCC 0.139922 0.028350 4.935
DOMY 0.002402 0.003983 0.603
PrevCM1 0.361076 0.087707 4.117
Correlation matrix not shown by default, as p = 13 > 12.
Use print(x, correlation=TRUE) or
vcov(x) if you need itcar::vif(mm0) GVIF Df GVIF^(1/(2*Df))
Tx 1.009521 2 1.002372
TestDIMcat20 1.003238 5 1.000323
Parity 1.141466 2 1.033631
DOSCC 1.319455 1 1.148676
DOMY 1.167395 1 1.080461
PrevCM 1.050427 1 1.024903
Step 4: Investigate effect measure modification
Tx:FARM (P > 0.05)
mm0 <- lmer(LSCC ~ Tx*FARMID + Parity + TestDIMcat20 + DOSCC + DOMY + PrevCM + (1|CowID), data=SDCTCOWDHI)
car::Anova(mm0)Analysis of Deviance Table (Type II Wald chisquare tests)
Response: LSCC
Chisq Df Pr(>Chisq)
Tx 0.3270 2 0.849151
FARMID 20.8837 6 0.001925 **
Parity 10.7805 2 0.004561 **
TestDIMcat20 153.2866 5 < 2.2e-16 ***
DOSCC 23.5900 1 1.192e-06 ***
DOMY 0.1511 1 0.697525
PrevCM 16.3381 1 5.299e-05 ***
Tx:FARMID 13.6785 12 0.321709
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
Tx:DIM category at herd test (P > 0.05)
mm0 <- lmer(LSCC ~ Tx*TestDIMcat20 + Parity + DOSCC + DOMY + PrevCM + (1|FARMID/CowID), data=SDCTCOWDHI)
car::Anova(mm0)Analysis of Deviance Table (Type II Wald chisquare tests)
Response: LSCC
Chisq Df Pr(>Chisq)
Tx 1.1452 2 0.56405
TestDIMcat20 155.5912 5 < 2.2e-16 ***
Parity 12.0313 2 0.00244 **
DOSCC 24.5071 1 7.404e-07 ***
DOMY 0.3514 1 0.55332
PrevCM 16.9005 1 3.939e-05 ***
Tx:TestDIMcat20 3.6320 10 0.96242
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
Tx:Parity (P > 0.05)
mm0 <- lmer(LSCC ~ Tx*Parity + TestDIMcat20 + DOSCC + DOMY + PrevCM + (1|FARMID/CowID), data=SDCTCOWDHI)
car::Anova(mm0)Analysis of Deviance Table (Type II Wald chisquare tests)
Response: LSCC
Chisq Df Pr(>Chisq)
Tx 1.1474 2 0.563433
Parity 12.0289 2 0.002443 **
TestDIMcat20 155.4765 5 < 2.2e-16 ***
DOSCC 25.0949 1 5.458e-07 ***
DOMY 0.4871 1 0.485238
PrevCM 16.3909 1 5.153e-05 ***
Tx:Parity 5.3325 4 0.254853
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
Tx:PrevCM (P > 0.05)
mm0 <- lmer(LSCC ~ Tx*PrevCM + Parity + TestDIMcat20 + DOSCC + DOMY + (1|FARMID/CowID), data=SDCTCOWDHI)
car::Anova(mm0)Analysis of Deviance Table (Type II Wald chisquare tests)
Response: LSCC
Chisq Df Pr(>Chisq)
Tx 1.1470 2 0.563549
PrevCM 16.9457 1 3.846e-05 ***
Parity 12.3024 2 0.002131 **
TestDIMcat20 155.7581 5 < 2.2e-16 ***
DOSCC 23.9345 1 9.967e-07 ***
DOMY 0.2841 1 0.594020
Tx:PrevCM 1.1795 2 0.554478
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
Tx:DOMY (P > 0.05)
mm0 <- lmer(LSCC ~ Tx*DOMY + PrevCM + Parity + TestDIMcat20 + DOSCC + (1|FARMID/CowID), data=SDCTCOWDHI)
car::Anova(mm0)Analysis of Deviance Table (Type II Wald chisquare tests)
Response: LSCC
Chisq Df Pr(>Chisq)
Tx 1.1481 2 0.56324
DOMY 0.3580 1 0.54964
PrevCM 17.4500 1 2.95e-05 ***
Parity 11.7208 2 0.00285 **
TestDIMcat20 155.5513 5 < 2.2e-16 ***
DOSCC 24.5607 1 7.20e-07 ***
Tx:DOMY 3.2738 2 0.19458
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1Step 5: Remove unneccesary covariates in backwards stepwise fashion using 10% rule
The order for removing covariates will be in increasing likelihood of being a confounder, which is based on my knowledge about the variables and their distribution in treatment groups.
This order will be: DOMY, Parity, PrevCM, DOSCC
TestDIMcat20 will not be removed.
Step 5a: Full model
mm0 <- lmer(LSCC ~ Tx + PrevCM + Parity + TestDIMcat20 + DOSCC + DOMY + (1|FARMID/CowID), data=SDCTCOWDHI)
mm0 %>% tidy()binding factor and character vector, coercing into character vectorbinding character and factor vector, coercing into character vector
Step 5b: removing DOMY
mm0 <- lmer(LSCC ~ Tx + PrevCM + Parity + TestDIMcat20 + DOSCC + (1|FARMID/CowID), data=SDCTCOWDHI)
mm0%>% tidy()binding factor and character vector, coercing into character vectorbinding character and factor vector, coercing into character vectorChanged by <10%. DOMY stays out.
Step 5b: removing Parity
mm0 <- lmer(LSCC ~ Tx + PrevCM + TestDIMcat20 + DOSCC + (1|FARMID/CowID), data=SDCTCOWDHI)
mm0 %>% tidy()binding factor and character vector, coercing into character vectorbinding character and factor vector, coercing into character vectorChanged by <10%. Parity stays out.
Step 5c: removing PrevCM
mm0 <- lmer(LSCC ~ Tx + TestDIMcat20 + DOSCC + (1|FARMID/CowID), data=SDCTCOWDHI)
mm0 %>% tidy()binding factor and character vector, coercing into character vectorbinding character and factor vector, coercing into character vectorChanged by >10%. PrevCM stays in.
Step 5d: removing DOSCC
mm0 <- lmer(LSCC ~ Tx + PrevCM + TestDIMcat20 + (1|FARMID/CowID), data=SDCTCOWDHI)
mm0 %>% tidy()binding factor and character vector, coercing into character vectorbinding character and factor vector, coercing into character vectorChanged by > 10%. DOSCC stays in.
Step 6a: Final model for SCC (1 - 120 DIM)
mm0 <- lmer(LSCC ~ Tx + PrevCM + DOSCC + TestDIMcat20 + (1|FARMID/CowID), data=SDCTCOWDHI)
mm0 %>% tidy(conf.int=TRUE)binding factor and character vector, coercing into character vectorbinding character and factor vector, coercing into character vectorICC for SCC (1 - 120 DIM)
mm0 <- lmer(LSCC ~ 1 + (1|FARMID/CowID), data=SDCTCOWDHI)
summary(mm0)Linear mixed model fit by REML ['lmerMod']
Formula: LSCC ~ 1 + (1 | FARMID/CowID)
Data: SDCTCOWDHI
REML criterion at convergence: 13293.7
Scaled residuals:
Min 1Q Median 3Q Max
-5.0159 -0.5107 -0.1062 0.4452 4.0999
Random effects:
Groups Name Variance Std.Dev.
CowID:FARMID (Intercept) 0.65871 0.8116
FARMID (Intercept) 0.03917 0.1979
Residual 1.20343 1.0970
Number of obs: 3989, groups: CowID:FARMID, 1178; FARMID, 7
Fixed effects:
Estimate Std. Error t value
(Intercept) 3.86908 0.08308 46.57ICC (CowID) = 0.35 ICC (FARMID) = 0.02
Most clustering is happening within cow, which is not a suprise given this is longitudinal data.
Step 6b: Final model for SCC (1 - 120 DIM) using P-value based backwards selection
lmer(LSCC ~ Tx + Parity + PrevCM + DOSCC + (1|FARMID/CowID), data=SDCTCOWDHI) %>% summary()Linear mixed model fit by REML ['lmerMod']
Formula: LSCC ~ Tx + Parity + PrevCM + DOSCC + (1 | FARMID/CowID)
Data: SDCTCOWDHI
REML criterion at convergence: 13233.2
Scaled residuals:
Min 1Q Median 3Q Max
-5.1671 -0.5100 -0.1122 0.4512 4.1458
Random effects:
Groups Name Variance Std.Dev.
CowID:FARMID (Intercept) 0.59501 0.7714
FARMID (Intercept) 0.01825 0.1351
Residual 1.20377 1.0972
Number of obs: 3989, groups: CowID:FARMID, 1178; FARMID, 7
Fixed effects:
Estimate Std. Error t value
(Intercept) 3.12018 0.13160 23.710
TxCulture 0.04063 0.06986 0.582
TxAlgorithm 0.06227 0.07063 0.882
Parity2 0.11211 0.07042 1.592
Parity3 0.25300 0.07477 3.384
PrevCM1 0.37041 0.08730 4.243
DOSCC 0.13267 0.02635 5.035
Correlation of Fixed Effects:
(Intr) TxCltr TxAlgr Party2 Party3 PrvCM1
TxCulture -0.298
TxAlgorithm -0.270 0.498
Parity2 -0.042 0.054 0.021
Parity3 0.038 0.012 0.019 0.439
PrevCM1 0.069 -0.019 0.013 -0.044 -0.100
DOSCC -0.786 0.026 0.000 -0.223 -0.295 -0.143lmer(LSCC ~ Tx + Parity + PrevCM + DOSCC + (1|FARMID/CowID), data=SDCTCOWDHI) %>% car::Anova() %>% tidy()NAEstimates are very similar to 10% rule based approach.
Step 6c: Final model (using 10% rule) reported as estimated marginal means (~LSmeans)
Geometric mean SCC
mm0 <- lmer(LSCC ~ Tx + PrevCM + DOSCC + TestDIMcat20 + (1|FARMID/CowID), data=SDCTCOWDHI)
emm <- emmeans(mm0, ~Tx) %>% tidy()Note: D.f. calculations have been disabled because the number of observations exceeds 3000.
To enable adjustments, set emm_options(pbkrtest.limit = 3989) or larger,
but be warned that this may result in large computation time and memory use.
Note: D.f. calculations have been disabled because the number of observations exceeds 3000.
To enable adjustments, set emm_options(lmerTest.limit = 3989) or larger,
but be warned that this may result in large computation time and memory use.emmStep 6c: Final model (using 10% rule) reported as back-transformed estimated marginal means (~LSmeans)
Geometric mean SCC
mm0 <- lmer(LSCC ~ Tx + PrevCM + DOSCC + TestDIMcat20 + (1|FARMID/CowID), data=SDCTCOWDHI)
emm <- emmeans(mm0, ~Tx) %>% tidy()Note: D.f. calculations have been disabled because the number of observations exceeds 3000.
To enable adjustments, set emm_options(pbkrtest.limit = 3989) or larger,
but be warned that this may result in large computation time and memory use.
Note: D.f. calculations have been disabled because the number of observations exceeds 3000.
To enable adjustments, set emm_options(lmerTest.limit = 3989) or larger,
but be warned that this may result in large computation time and memory use.emm$SCC <- exp(emm$estimate)
emm$LCL <- exp(emm$asymp.LCL)
emm$UCL <- exp(emm$asymp.UCL)
emm <- emm %>% select(Tx,SCC,LCL,UCL)
emm
Reported as back-transformed estimated marginal means by herd test day, no interaction with test DIM
mm0 <- lmer(LSCC ~ Tx + TestDIMcat20 + DOSCC + PrevCM + (1|FARMID/CowID), data=SDCTCOWDHI)
atx <- c(10,30,50,70,90,110)
emm <- emmeans(mm0, ~Tx*TestDIMcat20, at=list(atx)) %>% tidy()Note: D.f. calculations have been disabled because the number of observations exceeds 3000.
To enable adjustments, set emm_options(pbkrtest.limit = 3989) or larger,
but be warned that this may result in large computation time and memory use.
Note: D.f. calculations have been disabled because the number of observations exceeds 3000.
To enable adjustments, set emm_options(lmerTest.limit = 3989) or larger,
but be warned that this may result in large computation time and memory use.emm$SCC <- exp(emm$estimate)
emm$LCL <- exp(emm$asymp.LCL)
emm$UCL <- exp(emm$asymp.UCL)
emm <- emm %>% select(Tx,TestDIMcat20,SCC,LCL,UCL)
curve <- ggplot(emm) + coord_cartesian(ylim = (c(0,100))) + aes(x=TestDIMcat20, y=SCC, group=Tx, colour=Tx) + geom_point() + geom_line(aes(colour=Tx,linetype=Tx)) + geom_ribbon(aes(ymin=emm$LCL, ymax=emm$UCL,colour=Tx,fill=Tx), linetype=0, alpha=0.1)
curve
Reported as back-transformed estimated marginal means by herd test day, with interaction with test DIM
mm0 <- lmer(LSCC ~ Tx*TestDIMcat20 + PrevCM + DOSCC + (1|FARMID/CowID), data=SDCTCOWDHI)
atx <- c(10,30,50,70,90,110)
emm <- emmeans(mm0, ~Tx*TestDIMcat20, at=list(atx)) %>% tidy()Note: D.f. calculations have been disabled because the number of observations exceeds 3000.
To enable adjustments, set emm_options(pbkrtest.limit = 3989) or larger,
but be warned that this may result in large computation time and memory use.
Note: D.f. calculations have been disabled because the number of observations exceeds 3000.
To enable adjustments, set emm_options(lmerTest.limit = 3989) or larger,
but be warned that this may result in large computation time and memory use.emm$SCC <- exp(emm$estimate)
emm$LCL <- exp(emm$asymp.LCL)
emm$UCL <- exp(emm$asymp.UCL)
emm <- emm %>% select(Tx,TestDIMcat20,SCC,LCL,UCL)
curve <- ggplot(emm) + coord_cartesian(ylim = (c(0,100))) + aes(x=TestDIMcat20, y=SCC, group=Tx, colour=Tx) + geom_point() + geom_line(aes(colour=Tx,linetype=Tx)) + geom_ribbon(aes(ymin=emm$LCL, ymax=emm$UCL,colour=Tx,fill=Tx), linetype=0, alpha=0.1)
curve
### Step 7: Model diagnostics Checking homoskedasticity assumption (variance of residuals)
mm0 <- lmer(LSCC ~ Tx + TestDIMcat20 + (1|FARMID/CowID), data=SDCTCOWDHI)
ggplot(data.frame(eta=predict(mm0,type="link"),pearson=residuals(mm0,type="pearson")),
aes(x=eta,y=pearson)) +
geom_point() +
theme_bw()
Checking normality of residuals
qqnorm(residuals(mm0))
Evidence of small L tail. Although some evidence of heteroskedascitity and not perfectly normal residuals, I believe these are allowable for this model.
Outcome 4: Milk yield
Modelling plan
Model type: Linear mixed models, random intercepts for farm and cow will be fitted to account for repeated measures within cows, and clustering of cows within herds
Step 1: Identify potential confouders using a directed acyclic graph (DAG)
Step 2: Identify correlated variables using pearson and kendalls correlation coefficients
Step 3: Create model with all potential confounders
Step 4: Investigate potential effect measure modification
Step 5: Remove unneccesary covariates in backwards stepwise fashion using 10% rule (i.e. if estimated difference in milk yield changes by >10% after removing the covariate, the covariate is retained in the model)
Step 6: Report final model
Step 7: Model diagnostics
Step 1 & 2 Identify potential confounders using a DAG
library(DiagrammeR)
mermaid("graph LR
T(Treatment)-->U(Milk yield)
A(Age)-->T
P(Parity)-->T
M(Yield at dry-off)-->T
S(SCC during prev lactation)-->T
C(CM in prev lact)-->T
D(Days in milk at test)-->U
D-->T
A-->U
P-->U
M-->U
S-->U
C-->U
C-->M
P-->C
P-->S
P-->M
A-->P
A-->C
A-->S
A-->M
M-->S
C-->S
style A fill:#FFFFFF, stroke-width:0px
style T fill:#FFFFFF, stroke-width:2px
style P fill:#FFFFFF, stroke-width:0px
style M fill:#FFFFFF, stroke-width:0px
style S fill:#FFFFFF, stroke-width:0px
style C fill:#FFFFFF, stroke-width:0px
style I fill:#FFFFFF, stroke-width:0px
style D fill:#FFFFFF, stroke-width:0px
style U fill:#FFFFFF, stroke-width:2px
")According to this DAG, I may need to control for the following variables:
Parity [“Parity”] <- Age not offered as highly correlated
Yield at most recent test before dry off [“DOMY”]
Somatic cell count at last herd test during previous lactation [“DOSCC”] <- PrevSCCHI not offered as correlated
Clinical mastitis in previous lactation [“PrevCM”]
Days in milk at herd test (category, 0-20 “10”, 21-40 “30” etc) [“TestDIMcat20”]
Step 3: Create a model with all potential covariates
mm0 <- lmer(MY ~ Tx + TestDIMcat20 + Parity + DOSCC + PrevCM + DOMY + (1|FARMID/CowID), data=SDCTCOWDHI)
summary(mm0)Linear mixed model fit by REML ['lmerMod']
Formula: MY ~ Tx + TestDIMcat20 + Parity + DOSCC + PrevCM + DOMY + (1 | FARMID/CowID)
Data: SDCTCOWDHI
REML criterion at convergence: 27948.5
Scaled residuals:
Min 1Q Median 3Q Max
-6.3698 -0.4620 0.0223 0.5283 3.9851
Random effects:
Groups Name Variance Std.Dev.
CowID:FARMID (Intercept) 28.70 5.357
FARMID (Intercept) 9.63 3.103
Residual 46.18 6.795
Number of obs: 3990, groups: CowID:FARMID, 1177; FARMID, 7
Fixed effects:
Estimate Std. Error t value
(Intercept) 30.22473 1.76719 17.103
TxCulture -0.04144 0.46718 -0.089
TxAlgorithm -0.92795 0.47146 -1.968
TestDIMcat2030 11.86182 0.40548 29.253
TestDIMcat2050 12.87916 0.38854 33.148
TestDIMcat2070 12.51208 0.38249 32.713
TestDIMcat2090 11.01316 0.41572 26.492
TestDIMcat20110 8.43431 0.38401 21.964
Parity2 2.52549 0.47112 5.361
Parity3 3.01851 0.50067 6.029
DOSCC 0.26481 0.19029 1.392
PrevCM1 -0.24232 0.58620 -0.413
DOMY 0.22068 0.02706 8.156
Correlation matrix not shown by default, as p = 13 > 12.
Use print(x, correlation=TRUE) or
vcov(x) if you need itcar::vif(mm0) GVIF Df GVIF^(1/(2*Df))
Tx 1.009493 2 1.002365
TestDIMcat20 1.002987 5 1.000298
Parity 1.145893 2 1.034632
DOSCC 1.308164 1 1.143750
PrevCM 1.048252 1 1.023842
DOMY 1.154875 1 1.074651
Step 4: Investigate effect measure modification
Tx:FARM (P < 0.05)
mm0 <- lmer(MY ~ Tx*FARMID + Parity + TestDIMcat20 + DOSCC + DOMY + PrevCM + (1|CowID), data=SDCTCOWDHI)
car::Anova(mm0)Analysis of Deviance Table (Type II Wald chisquare tests)
Response: MY
Chisq Df Pr(>Chisq)
Tx 7.1919 2 0.02743 *
FARMID 383.1701 6 < 2.2e-16 ***
Parity 56.0941 2 6.597e-13 ***
TestDIMcat20 1502.4062 5 < 2.2e-16 ***
DOSCC 1.8384 1 0.17514
DOMY 73.6534 1 < 2.2e-16 ***
PrevCM 0.2286 1 0.63260
Tx:FARMID 21.1764 12 0.04786 *
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1Significant interaction term.
Descision: Revisit this after covariates are finalized.
Tx:DIM category at herd test (P > 0.05)
mm0 <- lmer(MY ~ Tx*TestDIMcat20 + Parity + DOSCC + DOMY + PrevCM + (1|FARMID/CowID), data=SDCTCOWDHI)
car::Anova(mm0)Analysis of Deviance Table (Type II Wald chisquare tests)
Response: MY
Chisq Df Pr(>Chisq)
Tx 4.9281 2 0.08509 .
TestDIMcat20 1563.6373 5 < 2.2e-16 ***
Parity 45.3091 2 1.450e-10 ***
DOSCC 1.8538 1 0.17335
DOMY 65.8173 1 4.947e-16 ***
PrevCM 0.1793 1 0.67194
Tx:TestDIMcat20 11.3951 10 0.32757
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
Tx:Parity (P > 0.05)
mm0 <- lmer(MY ~ Tx*Parity + TestDIMcat20 + DOSCC + DOMY + PrevCM + (1|FARMID/CowID), data=SDCTCOWDHI)
car::Anova(mm0)Analysis of Deviance Table (Type II Wald chisquare tests)
Response: MY
Chisq Df Pr(>Chisq)
Tx 4.9461 2 0.08433 .
Parity 45.4964 2 1.32e-10 ***
TestDIMcat20 1563.4543 5 < 2.2e-16 ***
DOSCC 2.2956 1 0.12974
DOMY 68.0948 1 < 2.2e-16 ***
PrevCM 0.2483 1 0.61826
Tx:Parity 6.9870 4 0.13658
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
Tx:PrevCM (P > 0.05)
mm0 <- lmer(MY ~ Tx*PrevCM + Parity + TestDIMcat20 + DOSCC + DOMY + (1|FARMID/CowID), data=SDCTCOWDHI)
car::Anova(mm0)Analysis of Deviance Table (Type II Wald chisquare tests)
Response: MY
Chisq Df Pr(>Chisq)
Tx 4.9345 2 0.08482 .
PrevCM 0.1709 1 0.67934
Parity 45.4975 2 1.319e-10 ***
TestDIMcat20 1561.6764 5 < 2.2e-16 ***
DOSCC 2.0081 1 0.15647
DOMY 67.3721 1 2.248e-16 ***
Tx:PrevCM 1.6370 2 0.44109
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
Step 5: Remove unnecessary covariates using backwards selection (10% rule)
The order for removing covariates will be in increasing likelihood of being a confounder, which is based on my knowledge about the variables and their distribution in treatment groups.
This order will be: DOSCC, PrevCM, Parity, DOMY
TestDIMcat20 will not be removed (forced into model).
Step 5a: Full model
mm0 <- lmer(MY ~ Tx + PrevCM + Parity + TestDIMcat20 + DOSCC + DOMY + (1|FARMID/CowID), data=SDCTCOWDHI)
mm0 %>% tidy() %>% select(term,estimate)binding factor and character vector, coercing into character vectorbinding character and factor vector, coercing into character vector
Step 5b: Removing DOSCC
mm0 <- lmer(MY ~ Tx + PrevCM + Parity + TestDIMcat20 + DOMY + (1|FARMID/CowID), data=SDCTCOWDHI)
mm0 %>% tidy() %>% select(term,estimate)binding factor and character vector, coercing into character vectorbinding character and factor vector, coercing into character vectorChanged by <10%. DOSCC stays out
Step 5c: removing PrevCM
mm0 <- lmer(MY ~ Tx + Parity + TestDIMcat20 + DOMY + (1|FARMID/CowID), data=SDCTCOWDHI)
mm0 %>% tidy() %>% select(term,estimate)binding factor and character vector, coercing into character vectorbinding character and factor vector, coercing into character vectorChanged by < 10%. PrevCM stays out.
Step 5d: removing Parity
mm0 <- lmer(MY ~ Tx + TestDIMcat20 + DOMY + (1|FARMID/CowID), data=SDCTCOWDHI)
mm0 %>% tidy() %>% select(term,estimate)binding factor and character vector, coercing into character vectorbinding character and factor vector, coercing into character vectorChanged by >10%. Parity stays in.
Step 5e: removing DOMY
mm0 <- lmer(MY ~ Tx + Parity + TestDIMcat20 + (1|FARMID/CowID), data=SDCTCOWDHI)
mm0 %>% tidy() %>% select(term,estimate)binding factor and character vector, coercing into character vectorbinding character and factor vector, coercing into character vectorChanged by >10%. DOMY stays in.
Revisit effect measure modification previously identified
Effect measure modification with herd (P < 0.05)
mm0 <- lmer(MY ~ Tx*FARMID + TestDIMcat20 + Parity + DOMY + (1|CowID), data=SDCTCOWDHI)
car::Anova(mm0)Analysis of Deviance Table (Type II Wald chisquare tests)
Response: MY
Chisq Df Pr(>Chisq)
Tx 7.0355 2 0.02967 *
FARMID 409.5611 6 < 2.2e-16 ***
TestDIMcat20 1502.2612 5 < 2.2e-16 ***
Parity 70.6857 2 4.475e-16 ***
DOMY 74.5582 1 < 2.2e-16 ***
Tx:FARMID 21.1553 12 0.04815 *
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1Decision - will investigate effect estimates stratified by herd
Step 6a: Final model for milk yield (1 - 120 DIM), stratified by herd.
Model output
mm0 <- lmer(MY ~ Tx*FARMID + TestDIMcat20 + Parity + DOMY + (1|CowID), data=SDCTCOWDHI)
emmeans(mm0,pairwise ~ Tx | FARMID)Note: D.f. calculations have been disabled because the number of observations exceeds 3000.
To enable adjustments, set emm_options(pbkrtest.limit = 3990) or larger,
but be warned that this may result in large computation time and memory use.
Note: D.f. calculations have been disabled because the number of observations exceeds 3000.
To enable adjustments, set emm_options(lmerTest.limit = 3990) or larger,
but be warned that this may result in large computation time and memory use.$emmeans
FARMID = 1:
Tx emmean SE df asymp.LCL asymp.UCL
Blanket 47.4 1.240 Inf 45.0 49.8
Culture 49.7 1.162 Inf 47.4 51.9
Algorithm 46.2 1.381 Inf 43.5 48.9
FARMID = 2:
Tx emmean SE df asymp.LCL asymp.UCL
Blanket 49.6 1.200 Inf 47.3 52.0
Culture 46.0 1.300 Inf 43.4 48.5
Algorithm 43.4 1.188 Inf 41.1 45.8
FARMID = 3:
Tx emmean SE df asymp.LCL asymp.UCL
Blanket 43.4 0.703 Inf 42.1 44.8
Culture 43.8 0.701 Inf 42.4 45.1
Algorithm 42.0 0.736 Inf 40.6 43.5
FARMID = 4:
Tx emmean SE df asymp.LCL asymp.UCL
Blanket 53.3 0.565 Inf 52.2 54.4
Culture 52.5 0.534 Inf 51.4 53.5
Algorithm 53.2 0.558 Inf 52.2 54.3
FARMID = 5:
Tx emmean SE df asymp.LCL asymp.UCL
Blanket 49.4 0.739 Inf 48.0 50.9
Culture 49.9 0.712 Inf 48.5 51.3
Algorithm 48.1 0.752 Inf 46.6 49.5
FARMID = 6:
Tx emmean SE df asymp.LCL asymp.UCL
Blanket 49.2 1.515 Inf 46.2 52.1
Culture 52.3 1.548 Inf 49.2 55.3
Algorithm 50.7 1.698 Inf 47.4 54.1
FARMID = 7:
Tx emmean SE df asymp.LCL asymp.UCL
Blanket 50.4 1.014 Inf 48.4 52.4
Culture 49.4 1.066 Inf 47.3 51.5
Algorithm 49.2 1.067 Inf 47.1 51.2
Results are averaged over the levels of: TestDIMcat20, Parity
Degrees-of-freedom method: asymptotic
Confidence level used: 0.95
$contrasts
FARMID = 1:
contrast estimate SE df z.ratio p.value
Blanket - Culture -2.2771 1.693 Inf -1.345 0.3704
Blanket - Algorithm 1.1734 1.848 Inf 0.635 0.8008
Culture - Algorithm 3.4506 1.800 Inf 1.917 0.1337
FARMID = 2:
contrast estimate SE df z.ratio p.value
Blanket - Culture 3.6627 1.767 Inf 2.072 0.0956
Blanket - Algorithm 6.2020 1.686 Inf 3.678 0.0007
Culture - Algorithm 2.5393 1.758 Inf 1.444 0.3182
FARMID = 3:
contrast estimate SE df z.ratio p.value
Blanket - Culture -0.3310 0.990 Inf -0.334 0.9402
Blanket - Algorithm 1.4086 1.015 Inf 1.388 0.3474
Culture - Algorithm 1.7396 1.015 Inf 1.713 0.2002
FARMID = 4:
contrast estimate SE df z.ratio p.value
Blanket - Culture 0.8679 0.770 Inf 1.128 0.4969
Blanket - Algorithm 0.0873 0.786 Inf 0.111 0.9932
Culture - Algorithm -0.7806 0.760 Inf -1.027 0.5600
FARMID = 5:
contrast estimate SE df z.ratio p.value
Blanket - Culture -0.5102 0.985 Inf -0.518 0.8627
Blanket - Algorithm 1.3592 1.015 Inf 1.339 0.3733
Culture - Algorithm 1.8695 1.003 Inf 1.863 0.1495
FARMID = 6:
contrast estimate SE df z.ratio p.value
Blanket - Culture -3.1044 2.164 Inf -1.435 0.3229
Blanket - Algorithm -1.5474 2.273 Inf -0.681 0.7747
Culture - Algorithm 1.5571 2.290 Inf 0.680 0.7753
FARMID = 7:
contrast estimate SE df z.ratio p.value
Blanket - Culture 0.9515 1.467 Inf 0.649 0.7931
Blanket - Algorithm 1.2379 1.468 Inf 0.843 0.6761
Culture - Algorithm 0.2864 1.503 Inf 0.191 0.9802
Results are averaged over the levels of: TestDIMcat20, Parity
P value adjustment: tukey method for comparing a family of 3 estimates It appears that herd 2 (87 cows) and herd 6 (42 cows) have extreme results compared with the other herds. In herd 2, BDCT had much higher MY than SDCT cows. In herd 6, it was the opposite effect. Given these were the smallest herds in the study, I will report a pooled result.
Step 6bi: Final model reported without herd interaction
Model output
summary(lmer(MY ~ Tx + TestDIMcat20 + Parity + DOMY+ (1|FARMID/CowID), data=SDCTCOWDHI))Linear mixed model fit by REML ['lmerMod']
Formula: MY ~ Tx + TestDIMcat20 + Parity + DOMY + (1 | FARMID/CowID)
Data: SDCTCOWDHI
REML criterion at convergence: 27949.8
Scaled residuals:
Min 1Q Median 3Q Max
-6.3374 -0.4572 0.0207 0.5284 3.9942
Random effects:
Groups Name Variance Std.Dev.
CowID:FARMID (Intercept) 28.677 5.355
FARMID (Intercept) 9.946 3.154
Residual 46.184 6.796
Number of obs: 3990, groups: CowID:FARMID, 1177; FARMID, 7
Fixed effects:
Estimate Std. Error t value
(Intercept) 31.61485 1.47112 21.490
TxCulture -0.04671 0.46702 -0.100
TxAlgorithm -0.92025 0.47129 -1.953
TestDIMcat2030 11.86555 0.40548 29.263
TestDIMcat2050 12.87773 0.38856 33.143
TestDIMcat2070 12.51168 0.38250 32.710
TestDIMcat2090 11.01773 0.41571 26.503
TestDIMcat20110 8.43423 0.38402 21.963
Parity2 2.66366 0.45780 5.818
Parity3 3.20325 0.47445 6.752
DOMY 0.20758 0.02539 8.175
Correlation of Fixed Effects:
(Intr) TxCltr TxAlgr TDIM203 TDIM205 TDIM207 TDIM209 TDIM201 Party2 Party3
TxCulture -0.132
TxAlgorithm -0.146 0.499
TstDIMc2030 -0.144 -0.013 -0.012
TstDIMc2050 -0.132 -0.010 -0.013 0.486
TstDIMc2070 -0.135 -0.010 -0.004 0.534 0.481
TstDIMc2090 -0.143 -0.005 -0.005 0.515 0.499 0.471
TstDIM20110 -0.129 -0.008 -0.004 0.514 0.507 0.527 0.465
Parity2 -0.167 0.055 0.021 0.000 0.008 -0.001 0.004 0.006
Parity3 -0.173 0.013 0.018 -0.014 0.009 -0.009 0.012 -0.001 0.399
DOMY -0.478 -0.065 -0.027 0.013 -0.002 -0.002 0.019 -0.014 0.066 0.101Model output (tidy)
lmer(MY ~ Tx + TestDIMcat20 + Parity + DOMY + (1|FARMID/CowID), data=SDCTCOWDHI) %>% tidy(conf.int=TRUE)binding factor and character vector, coercing into character vectorbinding character and factor vector, coercing into character vectorICC for MY (1 - 120 DIM)
mm0 <- lmer(MY ~ 1 + (1|FARMID/CowID), data=SDCTCOWDHI)
summary(mm0)Linear mixed model fit by REML ['lmerMod']
Formula: MY ~ 1 + (1 | FARMID/CowID)
Data: SDCTCOWDHI
REML criterion at convergence: 29310.2
Scaled residuals:
Min 1Q Median 3Q Max
-5.1211 -0.4867 0.0849 0.5681 3.5617
Random effects:
Groups Name Variance Std.Dev.
CowID:FARMID (Intercept) 27.03 5.199
FARMID (Intercept) 10.06 3.172
Residual 70.88 8.419
Number of obs: 3990, groups: CowID:FARMID, 1177; FARMID, 7
Fixed effects:
Estimate Std. Error t value
(Intercept) 48.286 1.226 39.38ICC (CowID) = 0.25 ICC (FARMID) = 0.09
Emmeans without TestDIM interaction
mm0 <- lmer(MY ~ Tx + TestDIMcat20 + Parity + DOMY+ (1|FARMID/CowID), data=SDCTCOWDHI)
emmeans(mm0,~Tx) %>% tidyNote: D.f. calculations have been disabled because the number of observations exceeds 3000.
To enable adjustments, set emm_options(pbkrtest.limit = 3990) or larger,
but be warned that this may result in large computation time and memory use.
Note: D.f. calculations have been disabled because the number of observations exceeds 3000.
To enable adjustments, set emm_options(lmerTest.limit = 3990) or larger,
but be warned that this may result in large computation time and memory use.Step 6bii: Final model reported without herd interaction (using P-value based backwards elimination)
Model output
summary(lmer(MY ~ Tx + TestDIMcat20 + Parity + (1|FARMID/CowID), data=SDCTCOWDHI))Linear mixed model fit by REML ['lmerMod']
Formula: MY ~ Tx + TestDIMcat20 + Parity + (1 | FARMID/CowID)
Data: SDCTCOWDHI
REML criterion at convergence: 28009.3
Scaled residuals:
Min 1Q Median 3Q Max
-6.2353 -0.4645 0.0246 0.5246 3.9839
Random effects:
Groups Name Variance Std.Dev.
CowID:FARMID (Intercept) 31.11 5.578
FARMID (Intercept) 11.14 3.338
Residual 46.17 6.795
Number of obs: 3990, groups: CowID:FARMID, 1177; FARMID, 7
Fixed effects:
Estimate Std. Error t value
(Intercept) 37.3558 1.3606 27.456
TxCulture 0.2008 0.4792 0.419
TxAlgorithm -0.8169 0.4844 -1.687
TestDIMcat2030 11.8422 0.4064 29.139
TestDIMcat2050 12.8803 0.3890 33.113
TestDIMcat2070 12.5248 0.3830 32.704
TestDIMcat2090 10.9596 0.4165 26.315
TestDIMcat20110 8.4713 0.3844 22.040
Parity2 2.4166 0.4697 5.145
Parity3 2.8034 0.4853 5.777
Correlation of Fixed Effects:
(Intr) TxCltr TxAlgr TDIM203 TDIM205 TDIM207 TDIM209 TDIM201 Party2
TxCulture -0.182
TxAlgorithm -0.177 0.499
TstDIMc2030 -0.150 -0.012 -0.012
TstDIMc2050 -0.144 -0.010 -0.013 0.486
TstDIMc2070 -0.147 -0.010 -0.004 0.535 0.480
TstDIMc2090 -0.145 -0.004 -0.005 0.515 0.499 0.470
TstDIM20110 -0.147 -0.009 -0.005 0.515 0.507 0.528 0.464
Parity2 -0.151 0.059 0.022 -0.001 0.008 -0.001 0.003 0.006
Parity3 -0.139 0.019 0.021 -0.015 0.009 -0.008 0.010 0.000 0.395
Step 6biii: Final model (using 10% rule) reported as estimated marginal means by test DIM category (no interaction with test DIM)
mm0 <- lmer(MY ~ Tx + TestDIMcat20 + Parity + DOMY + (1|FARMID/CowID), data=SDCTCOWDHI)
atx <- c(10,30,50,70,90,110)
emm <- emmeans(mm0, ~Tx*TestDIMcat20) %>% tidy()Note: D.f. calculations have been disabled because the number of observations exceeds 3000.
To enable adjustments, set emm_options(pbkrtest.limit = 3990) or larger,
but be warned that this may result in large computation time and memory use.
Note: D.f. calculations have been disabled because the number of observations exceeds 3000.
To enable adjustments, set emm_options(lmerTest.limit = 3990) or larger,
but be warned that this may result in large computation time and memory use.emm$MY <- emm$estimate
emm$LCL <- emm$asymp.LCL
emm$UCL <- emm$asymp.UCL
emm %>% select(Tx,TestDIMcat20,MY,LCL,UCL)Plotting model as predicted values
curve <- ggplot(emm) + coord_cartesian(ylim = (c(0,55))) + aes(x=TestDIMcat20, y=MY, group=Tx, colour=Tx) + geom_point() + geom_line(aes(colour=Tx,linetype=Tx)) + geom_ribbon(aes(ymin=emm$LCL, ymax=emm$UCL,colour=Tx,fill=Tx), linetype=0, alpha=0.1)
curve
Reported as estimated marginal means by test day category with interaction with herd test DIM
mm0 <- lmer(MY ~ Tx*TestDIMcat20 + Parity + DOMY + (1|FARMID/CowID), data=SDCTCOWDHI)
atx <- c(10,30,50,70,90,110)
emm <- emmeans(mm0, ~Tx*TestDIMcat20, at=list(atx)) %>% tidy()Note: D.f. calculations have been disabled because the number of observations exceeds 3000.
To enable adjustments, set emm_options(pbkrtest.limit = 3990) or larger,
but be warned that this may result in large computation time and memory use.
Note: D.f. calculations have been disabled because the number of observations exceeds 3000.
To enable adjustments, set emm_options(lmerTest.limit = 3990) or larger,
but be warned that this may result in large computation time and memory use.emm$MY <- emm$estimate
emm$LCL <- emm$asymp.LCL
emm$UCL <- emm$asymp.UCL
emm <- emm %>% select(Tx,TestDIMcat20,MY,LCL,UCL)
curve <- ggplot(emm) + coord_cartesian(ylim = (c(0,55))) + aes(x=TestDIMcat20, y=MY, group=Tx, colour=Tx) + geom_point() + geom_line(aes(colour=Tx,linetype=Tx)) + geom_ribbon(aes(ymin=emm$LCL, ymax=emm$UCL,colour=Tx,fill=Tx), linetype=0, alpha=0.1)
curve
Step 7: Model diagnostics
Checking homoskedasticity assumption (variance of residuals)
mm0 <- lmer(MY ~ Tx + TestDIMcat20 + Parity +DOMY+ (1|FARMID/CowID), data=SDCTCOWDHI)
ggplot(data.frame(eta=predict(mm0,type="link"),pearson=residuals(mm0,type="pearson")),
aes(x=eta,y=pearson)) +
geom_point() +
theme_bw()
Checking normality of residuals
qqnorm(residuals(mm0))
I am happy with homoscedasticity and normal residuals assumptions
LS0tCnRpdGxlOiAiU2VsZWN0aXZlIERyeSBDb3cgVGhlcmFweTogQ293LWxldmVsIGFuYWx5c2lzIgpvdXRwdXQ6CiAgaHRtbF9ub3RlYm9vazoKICAgIHRvYzogdHJ1ZQogICAgdG9jX2RlcHRoOiAzCiAgICB0b2NfZmxvYXQ6CiAgICAgIGNvbGxhcHNlZDogdHJ1ZQoKLS0tCgpgYGB7ciBzZXR1cH0KI01hYwprbml0cjo6b3B0c19rbml0JHNldChyb290LmRpciA9ICJ+L0Ryb3Bib3gvUiBiYWNrdXAvU0RDVCAtIFIiKQoKI1dpbmRvd3MKI2tuaXRyOjpvcHRzX2tuaXQkc2V0KHJvb3QuZGlyID0gIkM6L1VzZXJzL3Jvd2UwMTIyL0Ryb3Bib3gvUiBiYWNrdXAvU0RDVCAtIFIiKQoKYGBgCgoKYGBge3J9CmxpYnJhcnkoa25pdHIpCmBgYAoKTG9hZCBkYXRhCgpgYGB7cn0KbG9hZChmaWxlPSJCYXNlbGluZS5SZGF0YSIpCmxvYWQoZmlsZT0iU0RDVENPVy5SZGF0YSIpCmxvYWQoZmlsZT0iU0RDVENPV0RISS5SZGF0YSIpCgpTRENUQ09XID0gU0RDVENPVyAlPiUKICBtdXRhdGUoVHggPSByZWNvZGUoU0RDVENPVyRUeCwKICAgICAgICAgICAgICAgICAgICAgIjAiID0gIkJsYW5rZXQiLCAiMSIgPSAiQ3VsdHVyZSIsICIyIiA9ICJBbGdvcml0aG0iKSkKCmBgYAoKCkltcG9ydCBmaWxlIEJMIChjb3dzIGluY2x1ZGVkIGZvciBhbmFseXNpcykgZm9yIGRlc2NyaXB0aXZlIHN0YXRpc3RpY3MKYGBge3J9CmxpYnJhcnkocmVhZHIpCkJMIDwtIHJlYWRfY3N2KCJCTC5jc3YiLCBjb2xfdHlwZXMgPSBjb2xzKFgxID0gY29sX3NraXAoKSkpCmhlYWQoQkwpCmBgYAojIEluc3BlY3QgZGF0YQoKYGBge3J9CnByaW50KHN1bW1hcnl0b29sczo6ZGZTdW1tYXJ5KEJMLCB2YWxpZC5jb2w9RkFMU0UsIGdyYXBoLm1hZ25pZj0wLjgsIHN0eWxlPSJncmlkIiksIG1ldGhvZCA9ICJyZW5kZXIiKQpgYGAKCiMjIERlc2NyaXB0aXZlIHN0YXRpc3RpY3Mgb2Ygc3ViamVjdHMgYXQgZW5yb2xsbWVudApDb21wYXJpc29uIG9mIGRlbW9ncmFwaGljcyBmb3IgZWFjaCB0cmVhdG1lbnQgZ3JvdXAgYXQgZHJ5LW9mZgpgYGB7cn0KbGlicmFyeSh0YWJsZTEpCnRhYmxlMSh+IEFnZSArIERPTVkgKyBET1NDQyArIFByZXZTQ0NISSArIGZhY3RvcihQcmV2Q00pICsgZmFjdG9yKFBhcml0eSkgfCBUeCwgZGF0YT1CYXNlbGluZSkKYGBgCgojIyBEZXNjcmlwdGl2ZSBzdGF0aXN0aWNzIG9mIHN1YmplY3RzIGF0IGRyeS1vZmYKTm90ZSB0aGF0IHRoaXMgdGFibGUgZXhjbHVkZXMgY293cyAobj0zMikgdGhhdCB3ZXJlIG5vdCBpbmNsdWRlZCBjb3ctbGV2ZWwgYW5hbHlzaXMgKGVnLiBjb3dzIHdpdGggbG9uZyBvciBzaG9ydCBkcnkgcGVyaW9kcykuCmBgYHtyfQp0YWJsZTEofiBBZ2UgKyBET01ZICsgRFBsZW5ndGggKyBET1NDQyArIFByZXZTQ0NISSArIFByZXZDTSArIFBhcml0eSB8IFR4LCBkYXRhPVNEQ1RDT1cpCmBgYApHcm91cHMgYXBwZWFyIHRvIGJlIHdlbGwgYmFsYW5jZWQgYmVmb3JlIGFuZCBhZnRlciBleGNsdXNpb25zLCBpbmRpY2F0aW5nIHRoYXQgcmFuZG9taXphdGlvbiB3YXMgc3VjY2Vzc2Z1bCwgYW5kIHRoYXQgZHJvcC1vdXQgaXMgdW5saWtlbHkgdG8gYWZmZWN0IG91ciBhc3N1bXB0aW9uIG9mIGV4Y2hhbmdlYWJpbGl0eSBiZXR3ZWVuIHRyZWF0bWVudCBncm91cHMuCgo8YnI+Cjxicj4KPGJyPgo8YnI+CkNvbXBhcmlzb24gb2YgaGVyZHMKYGBge3J9CmxpYnJhcnkodGFibGUxKQp0YWJsZTEofiBUeCArIEFnZSArIERPTVkgKyBEUGxlbmd0aCArIERPRElNICsgRE9TQ0MgKyBQcmV2U0NDSEkgKyBQcmV2Q00gKyBQYXJpdHkgfCBGQVJNSUQsIGRhdGE9U0RDVENPVykKYGBgCgoKIyBPdXRjb21lIDE6IENsaW5pY2FsIG1hc3RpdGlzCiMjIEthcGxhbiBNZWllciBjdXJ2ZSAmIGxvZyByYW5rIHRlc3QgZm9yIGNsaW5pY2FsIG1hc3RpdGlzCgpLYXBsYW4gTWVpZXIgY3VydmUKYGBge3IgZmlnLmhlaWdodD04fQpsaWJyYXJ5KGdncGxvdDIpCmxpYnJhcnkoc3Vydm1pbmVyKQpLTSA8LSBzdXJ2Zml0KFN1cnYoQ01UQVIsIENNKSB+IFR4LCBkYXRhID0gU0RDVENPVykKCmtuaXRyOjpvcHRzX2NodW5rJHNldChmaWcud2lkdGggPSA4MDAsIGZpZy5oZWlnaHQgPSA5MDApCmdnc3VydnBsb3QoS00sIGRhdGEgPSBTRENUQ09XLCAgdGl0bGUgPSAiIiwgcHZhbCA9IFQsIGNvbmYuaW50ID0gRixyaXNrLnRhYmxlLmNvbCA9ICJUeCIscmlzay50YWJsZSA9IFQsIHJpc2sudGFibGUueS50ZXh0LmNvbCA9IFRSVUUgLCBzdXJ2LnBsb3QuaGVpZ2h0ID0gNSwgbGVnZW5kLmxhYnMgPSBjKCJCbGFua2V0IiwiQ3VsdHVyZSIsIkFsZ29yaXRobSIpLCB0YWJsZXMudGhlbWUgPSB0aGVtZV9jbGVhbnRhYmxlKCksIGdndGhlbWUgPSB0aGVtZV9idygpKQoKYGBgCjxicj4KPGJyPgo8YnI+Cjxicj4KTG9nLVJhbmsgdGVzdApgYGB7cn0Kc3VydmRpZmYoU3VydihDTVRBUiwgQ00pIH4gVHgsZGF0YT1TRENUQ09XLHJobz0wKQpgYGAKCgojIyBDb3ggcHJvcG9ydGlvbmFsIGhhemFyZHMgcmVncmVzc2lvbiBmb3IgY2xpbmljYWwgbWFzdGl0aXMgKDEtMTIwIERJTSkKIyMgTW9kZWwgYnVpbGRpbmcgcGxhbgoKTW9kZWwgdHlwZTogQ294IHByb3BvcnRpb25hbCBoYXphcmRzIGFuYWx5c2lzIHdpdGggZmFybSBpbmNsdWRlZCBhcyBhIGNsdXN0ZXIgdmFyaWFibGUgKHJvYnVzdCBzYW5kd2ljaCBzdGFuZGFyZCBlcnJvciBlc3RpbWF0b3IpIHRvIGFjY291bnQgZm9yIGxhY2sgb2YgaW5kZXBlZGVuY2UuIAoKU3RlcCAxOiBJZGVudGlmeSBwb3RlbnRpYWwgY29uZm91ZGVycyB1c2luZyBhIGRpcmVjdGVkIGFjeWNsaWMgZ3JhcGggKERBRykKClN0ZXAgMjogSWRlbnRpZnkgY29ycmVsYXRlZCB2YXJpYWJsZXMgdXNpbmcgcGVhcnNvbiBhbmQga2VuZGFsbHMgY29ycmVsYXRpb24gY29lZmZpY2llbnRzCgpTdGVwIDM6IENyZWF0ZSBtb2RlbCB3aXRoIGFsbCBwb3RlbnRpYWwgY29uZm91bmRlcnMKClN0ZXAgNDogSW52ZXN0aWdhdGUgaWYgY292YXJpYXRlcyBtZWV0IHByb3BvcnRpb25hbCBoYXphcmRzIGFzc3VtcHRpb24KClN0ZXAgNTogSW52ZXN0aWdhdGUgcG90ZW50aWFsIGVmZmVjdCBtZWFzdXJlIG1vZGlmaWNhdGlvbgoKU3RlcCA2OiBSZW1vdmUgdW5uZWNjZXNhcnkgY292YXJpYXRlcyBpbiBiYWNrd2FyZHMgc3RlcHdpc2UgZmFzaGlvbiB1c2luZyAxMCUgcnVsZSAoaS5lLiBpZiBoYXphcmQgcmF0aW8gZm9yIGFsZ29yaXRobSBvciBjdWx0dXJlIGNoYW5nZXMgYnkgPjEwJSBhZnRlciByZW1vdmluZyB0aGUgY292YXJpYXRlLCB0aGUgY292YXJpYXRlIGlzIHJldGFpbmVkIGluIHRoZSBtb2RlbCkKClN0ZXAgNzogUmVwb3J0IGZpbmFsIG1vZGVsCgojIyMgU3RlcCAxOiBEQUcgZm9yIGNsaW5jYWwgbWFzdGl0aXMKVGhpcyBpcyB1c2VkIHRvIGlkZW50aWZ5IHZhcmlhYmxlcyB0aGF0IGNvdWxkIGJlIGNvbmZvdW5kZXJzIGlmIHRoZXkgYXJlIG5vdCBiYWxhbmNlZCBiZXR3ZWVuIHRyZWF0bWVudCBncm91cHMuIApgYGB7ciBmaWcud2lkdGggPSAxMH0KbGlicmFyeShEaWFncmFtbWVSKQptZXJtYWlkKCJncmFwaCBMUgogICAgICAgIFQoVHJlYXRtZW50KS0tPlUoQ2xpbmljYWwgbWFzdGl0aXMpCiAgICAgICAgQShBZ2UpLS0+VAogICAgICAgIFAoUGFyaXR5KS0tPlQKICAgICAgICBNKFlpZWxkIGF0IGRyeS1vZmYpLS0+VAogICAgICAgIFMoU0NDIGR1cmluZyBwcmV2IGxhY3RhdGlvbiktLT5UCiAgICAgICAgQyhDTSBpbiBwcmV2IGxhY3QpLS0+VAogICAgICAgIEEtLT5VCiAgICAgICAgUC0tPlUKICAgICAgICBNLS0+VQogICAgICAgIFMtLT5VCiAgICAgICAgQy0tPlUKICAgICAgICBDLS0+TQogICAgICAgIFAtLT5DCiAgICAgICAgUC0tPlMKICAgICAgICBQLS0+TQogICAgICAgIEEtLT5QCiAgICAgICAgQS0tPkMKICAgICAgICBBLS0+UwogICAgICAgIEEtLT5NCiAgICAgICAgTS0tPlMKICAgICAgICBDLS0+UwogICAgICAgIHN0eWxlIEEgZmlsbDojRkZGRkZGLCBzdHJva2Utd2lkdGg6MHB4CiAgICAgICAgc3R5bGUgVCBmaWxsOiNGRkZGRkYsIHN0cm9rZS13aWR0aDoycHgKICAgICAgICBzdHlsZSBQIGZpbGw6I0ZGRkZGRiwgc3Ryb2tlLXdpZHRoOjBweAogICAgICAgIHN0eWxlIE0gZmlsbDojRkZGRkZGLCBzdHJva2Utd2lkdGg6MHB4CiAgICAgICAgc3R5bGUgUyBmaWxsOiNGRkZGRkYsIHN0cm9rZS13aWR0aDowcHgKICAgICAgICBzdHlsZSBDIGZpbGw6I0ZGRkZGRiwgc3Ryb2tlLXdpZHRoOjBweAogICAgICAgIHN0eWxlIEkgZmlsbDojRkZGRkZGLCBzdHJva2Utd2lkdGg6MHB4CiAgICAgICAgc3R5bGUgVSBmaWxsOiNGRkZGRkYsIHN0cm9rZS13aWR0aDoycHgKICAgICAgICAiKQpgYGAKQWNjb3JkaW5nIHRvIHRoaXMgREFHLCBJIG1heSBuZWVkIHRvIGNvbnRyb2wgZm9yIHRoZSBmb2xsb3dpbmcgdmFyaWFibGVzLgoKUGFyaXR5IFsiUGFyaXR5Il0KCkFnZSBbIkFnZSJdCgpZaWVsZCBhdCBtb3N0IHJlY2VudCB0ZXN0IGJlZm9yZSBkcnkgb2ZmIFsiRE9NWSJdCgpTb21hdGljIGNlbGwgY291bnQgYXQgbGFzdCBoZXJkIHRlc3QgZHVyaW5nIHByZXZpb3VzIGxhY3RhdGlvbiBbIkRPU0NDIiBvciAiUHJldlNDQ0hpIl0gPC0gbGlrZWx5IHRvIGJlIGNvcnJlbGF0ZWQKCkNsaW5pY2FsIG1hc3RpdGlzIGluIHByZXZpb3VzIGxhY3RhdGlvbiBbIlByZXZDTSJdCgoKPGJyPgo8YnI+Cjxicj4KPGJyPgoKIyMjIFN0ZXAgMjogSWRlbnRpZnkgY29ycmVsYXRlZCBjb3ZhcmlhdGVzClBlYXJzb24gY29ycmVsYXRpb24gbWF0cml4IGFtb25nIHBvdGVudGlhbCBwcmVkaWN0b3JzCmBgYHtyfQpjb3IgPC0gQkxbLCBjKDQsNSw2LDcsOSwxMCldCmNvciA8LSBjb3IoY29yLCB1c2UgPSAiY29tcGxldGUub2JzIikKcm91bmQoY29yLCAyKQpgYGAKCmBgYHtyfQpjb3IgPC0gQkxbLCBjKDQsNSw2LDcsOSwxMCldCmNvciA8LSBjb3IoY29yLCB1c2UgPSAiY29tcGxldGUub2JzIiwgbWV0aG9kPSJrZW5kYWwiKQpyb3VuZChjb3IsIDIpCmBgYAoKQWdlIGFuZCBQYXJpdHkgaGlnaGx5IGNvcnJlbGF0ZWQgYXMgZXhwZWN0ZWQuICBXaWxsIG9ubHkgb2ZmZXIgcGFyaXR5LiAKClByZXZpb3VzIGxhY3RhdGlvbiBwZWFrIFNDQyBpcyBtb2RlcmF0ZWx5IGNvcnJlbGF0ZWQgd2l0aCBTQ0MgYXQgbGFzdCBoZXJkIHRlc3QgKERPU0NDKS4gV2lsbCBvbmx5IG9mZmVyIERPU0NDLiAKCjxicj4KPGJyPgo8YnI+Cjxicj4KCiMjIyBTdGVwIDM6IENyZWF0ZSBtb2RlbCB3aXRoIGFsbCBwb3RlbnRpYWwgY292YXJpYXRlcwpDb3ggcHJvcG9ydGlvbmFsIGhhemFyZHMgcmVncmVzc2lvbiBmb3IgY2xpbmljYWwgbWFzdGl0aXMgKDEtMTIwIERJTSkKCkNvd3MgdGhhdCBkaWQgbm90IGNhbHZlIG9yIHRoYXQgaGFkIGxvbmcvc2hvcnQgZHJ5IHBlcmlvZHMgYXJlIGV4Y2x1ZGVkLiAKQ293cyB3aXRoIGNsaW5pY2FsIG1hc3RpdGlzIHByaW9yIHRvIGNhbHZpbmcgYXJlIGluY2x1ZGVkLiAKClJlYXNvbnMgZm9yIFIgY2Vuc29yID0gMTIwIERJTSBvciBjdWxsaW5nL2RlYXRoCgpgYGB7cn0KbGlicmFyeShicm9vbSkKbGlicmFyeShzdXJ2aXZhbCkKU1IgPC0gY294cGgoU3VydihDTVRBUiwgQ00pIH4gUGFyaXR5ICsgRE9NWSArIERPU0NDICsgUHJldkNNICsgVHggKyBjbHVzdGVyKEZBUk1JRCksIGRhdGE9U0RDVENPVykKY294cGgoU3VydihDTVRBUiwgQ00pIH4gUGFyaXR5ICsgRE9NWSArIERPU0NDICsgUHJldkNNICsgVHggKyBjbHVzdGVyKEZBUk1JRCksIGRhdGE9U0RDVENPVykKCiN0aWR5KGNveHBoKFN1cnYoQ01UQVIsIENNKSB+IFR4ICsgY2x1c3RlcihGQVJNSUQpLCBkYXRhPVNEQ1RDT1cpKQpgYGAKCmBgYHtyfQpsaWJyYXJ5KGJyb29tKQpsaWJyYXJ5KHN1cnZpdmFsKQpTUiA8LSBjb3hwaChTdXJ2KENNVEFSLCBDTSkgfiBQYXJpdHkgKyBET01ZICsgRE9TQ0MgKyBQcmV2Q00gKyBUeCArIGNsdXN0ZXIoRkFSTUlEKSwgZGF0YT1TRENUQ09XKQpjb3hwaChTdXJ2KENNVEFSLCBDTSkgfiBQYXJpdHkgKyBET01ZICsgRE9TQ0MgKyBQcmV2Q00gKyBUeCArIGNsdXN0ZXIoRkFSTUlEKSwgZGF0YT1TRENUQ09XKQoKI3RpZHkoY294cGgoU3VydihDTVRBUiwgQ00pIH4gVHggKyBjbHVzdGVyKEZBUk1JRCksIGRhdGE9U0RDVENPVykpCmBgYAoKIyMjIFN0ZXAgNDogU2Nob2VuZmllbGQgdGVzdHMgdG8gYXNzZXNzIGFzc3VtcHRpb24gb2YgcHJvcG9ydGlvbmFsIGhhemFyZHMgZm9yIGV4cGxhbmF0b3J5IHZhcmlhYmxlcy4KClBhcml0eSwgVHggYW5kIERPU0NDIHZpb2xhdGUgdGhlIGFzc3VtcHRpb24gb2YgcHJvcG9ydGlvbmFsIGhhemFyZHMKYGBge3J9ClNSIDwtIGNveC56cGgoU1IpClNSCmBgYAo8YnI+Cjxicj4KPGJyPgo8YnI+CgpTY2hvZW5maWVsZCByZXNpZHVhbCBwbG90cyB0byBhc3Nlc3MgYXNzdW1wdGlvbiBvZiBwcm9wb3J0aW9uYWwgaGF6YXJkcy4gCmBgYHtyIGZpZy5oZWlnaHQ9OH0KZ2djb3h6cGgoU1IsdmFyID0gYygiVHhDdWx0dXJlIiwiVHhBbGdvcml0aG0iKSkKYGBgCgpgYGB7ciBmaWcuaGVpZ2h0PTh9CmdnY294enBoKFNSLHZhciA9IGMoIkRPU0NDIiwgIkRPTVkiKSkKYGBgCmBgYHtyIGZpZy5oZWlnaHQ9OH0KZ2djb3h6cGgoU1IsdmFyID0gYygiUGFyaXR5MiIsIlBhcml0eTMiKSkKYGBgCmBgYHtyIGZpZy5oZWlnaHQ9OH0KZ2djb3h6cGgoU1IsdmFyID0gYygiUHJldkNNMSIpKQpgYGAKCjxicj4KPGJyPgo8YnI+Cjxicj4KSSB3aWxsIGRlYWwgd2l0aCB0aGVzZSBjb3ZhcmlhdGVzIHVzaW5nIHN0cmF0aWZpY2F0aW9uIChpLmUuIGZpdHRpbmcgc2VwZXJhdGUgYmFzZWxpbmUgaGF6YXJkcykKCkkgbXVzdCBjcmVhdGUgY2F0ZWdvcmljYWwgdmFyaWFibGVzIHRvIGFsbG93IGZvciB0aGlzIChub3Qgc2hvd24gaGVyZSkuIAoKRHJ5IG9mZiBtaWxrIHlpZWxkIFsiRE9NWSJdIC0+IG1lZGlhbiBzcGxpdCAoIkRPTVljYXQiID0gMCAvIDEpLgoKRHJ5IG9mZiBTQ0MgLSBzcGxpdCBhdCAyMDAsMDAwIGNlbGxzIChzdWJjbGluaWNhbCBtYXN0aXRpcyBbIkRPU0NNIl0gPSAwIC8gMSkKCjxicj4KPGJyPgo8YnI+Cjxicj4KU2hvdyBuZXcgdmFyaWFibGVzCmBgYHtyfQpTRENUQ09XY2hlY2sgPC0gU0RDVENPVyAlPiUgc2VsZWN0KFR4LENNLENNVEFSLEN1bGwyLEN1bGwyVEFSLERPU0NDLERPU0NNLERPTVksRE9NWWNhdCxQYXJpdHksUHJldkNNKQpoZWFkKFNEQ1RDT1djaGVjaykKYGBgCjxicj4KPGJyPgo8YnI+Cjxicj4KClJ1biBuZXcgY294IG1vZGVsIHdpdGggc3RyYXRpZmllZCB2YXJpYWJsZXMgCmBgYHtyfQpTUiA8LSBjb3hwaChTdXJ2KENNVEFSLCBDTSkgfiBzdHJhdGEoUGFyaXR5KSArIHN0cmF0YShET01ZY2F0KSArIHN0cmF0YShET1NDTSkgKyBQcmV2Q00gKyBUeCArIGNsdXN0ZXIoRkFSTUlEKSwgZGF0YT1TRENUQ09XKQp0aWR5KGNveHBoKFN1cnYoQ01UQVIsIENNKSB+IHN0cmF0YShQYXJpdHkpICsgc3RyYXRhKERPTVljYXQpICsgc3RyYXRhKERPU0NNKSArIFByZXZDTSArIFR4ICsgY2x1c3RlcihGQVJNSUQpLCBkYXRhPVNEQ1RDT1cpKQpgYGAKPGJyPgo8YnI+Cjxicj4KPGJyPgpSZXBlYXQgU2Nob2VuZmVsZCByZXNpZHVhbHMKClR4IG5vIGxvbmdlciBwIDwgMC4wNSwgYnV0IGlzIGNsb3NlIHRvIHRoZSB0aHJlc2hvbGQgKHAgPSAwLjA4KS4KYGBge3J9ClNSIDwtIGNveC56cGgoU1IpClNSCmBgYAo8YnI+Cjxicj4KPGJyPgo8YnI+CldpbGwgc3BsaXQgdGltZSBpbnRvIDEtNjAgYW5kIDYxLTEyMCBESU0gdG8gc2VlIGlmIHRoaXMgZGlmZmVyZW5jZSBpcyBtZWFuaW5nZnVsLgoKQ294IG1vZGVsIDEtNjAgRElNCgpCZXRhIGNvZWZmaWNpZW50IGZvciBDdWx0dXJlID0gLTAuNDMsIEFsZ29yaXRobSA9IC0wLjA4CmBgYHtyfQpTUiA8LSBjb3hwaChTdXJ2KENNVEFSNjAsIENNNjApIH4gc3RyYXRhKFBhcml0eSkgKyBzdHJhdGEoRE9NWWNhdCkgKyBzdHJhdGEoRE9TQ00pICsgUHJldkNNICsgVHggKyBjbHVzdGVyKEZBUk1JRCksIGRhdGE9U0RDVENPVykKdGlkeShjb3hwaChTdXJ2KENNVEFSNjAsIENNNjApIH4gc3RyYXRhKFBhcml0eSkgKyBzdHJhdGEoRE9NWWNhdCkgKyBzdHJhdGEoRE9TQ00pICsgUHJldkNNICsgVHggKyBjbHVzdGVyKEZBUk1JRCksIGRhdGE9U0RDVENPVykpCmBgYAo8YnI+Cjxicj4KPGJyPgo8YnI+CkNveCBtb2RlbCA2MS0xMjAgRElNCkJldGEgY29lZmZpY2llbnQgZm9yIEN1bHR1cmUgPSAwLjAwOCwgQWxnb3JpdGhtID0gLS40MDEKYGBge3J9ClNSIDwtIGNveHBoKFN1cnYoQ01UQVI2MDEyMCwgQ002MDEyMCkgfiBzdHJhdGEoUGFyaXR5KSArIHN0cmF0YShET01ZY2F0KSArIHN0cmF0YShET1NDTSkgKyBQcmV2Q00gKyBUeCArIGNsdXN0ZXIoRkFSTUlEKSwgZGF0YT1TRENUQ09XKQp0aWR5KGNveHBoKFN1cnYoQ01UQVI2MDEyMCwgQ002MDEyMCkgfiBzdHJhdGEoUGFyaXR5KSArIHN0cmF0YShET01ZY2F0KSArIHN0cmF0YShET1NDTSkgKyBQcmV2Q00gKyBUeCArIGNsdXN0ZXIoRkFSTUlEKSwgZGF0YT1TRENUQ09XKSkKYGBgCgpEZWNpc2lvbjogIFRoZXNlIGRpZmZlcmVuY2VzIGFyZW4ndCBzdWZmaWNpZW50IGluIG15IG9waW5pb24gdG8gd2FycmFudCBwdXJzdWluZyB0aGUgMS02MCwgNjEtMTIwIERJTSBtb2RlbHMuIEFuIGF2ZXJhZ2Ugb2YgdGhlIHR3byBpcyBhcHByb3ByaWF0ZS4KPGJyPgo8YnI+Cjxicj4KPGJyPgoKIyMjIFN0ZXAgNTogSW52ZXN0aWdhdGUgZWZmZWN0IG1lYXN1cmUgbW9kaWZpY2F0aW9uCgpVbmFibGUgdG8gZXZsdWF0ZSBpbnRlcmFjdGlvbiBiZXR3ZWVuIFR4ICYgRkFSTUlEIHVzaW5nIENveCByZWdyZXNzaW9uIC0gQml6emFyZSBTRSdzLCB3b24ndCBjb252ZXJnZS4gCmBgYHtyfQptbTAgPC0gY294cGgoU3VydihDTVRBUiwgQ00pIH4gVHggKyBGQVJNSUQgKyBUeDpGQVJNSUQgKyBQcmV2Q00gKyBzdHJhdGEoUGFyaXR5KSArIHN0cmF0YShET01ZY2F0KSArIHN0cmF0YShET1NDTSksIGRhdGE9U0RDVENPVykKdGlkeShtbTApCmBgYAo8YnI+Cjxicj4KPGJyPgo8YnI+CldpbGwgdXNlIGxvZ2lzdGljIHJlZ3Jlc3Npb24gdG8gYXNzZXNzIGZvciBpbnRlcmFjdGlvbiBiZXR3ZWVuIFR4IGFuZCBmYXJtOiBQID4gMC4wNS4gCmBgYHtyfQptbTAgPC0gZ2xtKENNIH4gVHgqRkFSTUlEICsgUHJldkNNICsgUGFyaXR5ICsgRE9NWWNhdCArIERPU0NNLCBkYXRhPVNEQ1RDT1csIGZhbWlseT1iaW5vbWlhbCkKY2FyOjpBbm92YShtbTApCmBgYAoKRGVjaXNpb246IE5vIHRyZWF0bWVudCBieSBoZXJkIGVmZmVjdCBtZWFzdXJlIG1vZGlmaWNhdGlvbgoKPGJyPgo8YnI+Cjxicj4KPGJyPgpBc3Nlc3MgZWZmZWN0IG1lYXN1cmUgbW9kaWZpY2F0aW9uIHdpdGggcHJldmlvdXMgY2xpbmljYWwgbWFzdGl0aXMgdXNpbmcgQ294IHJlZ3Jlc3Npb24uIApgYGB7cn0KCmEgPC0gY294cGgoU3VydihDTVRBUiwgQ00pIH4gZmFjdG9yKFR4KSpmYWN0b3IoUHJldkNNKSArIHN0cmF0YShQYXJpdHkpICsgY2x1c3RlcihGQVJNSUQpICsgc3RyYXRhKERPTVljYXQpICsgc3RyYXRhKERPU0NNKSwgZGF0YT1TRENUQ09XKQoKbGlicmFyeShhb2QpCndhbGQudGVzdChiID0gY29lZihhKSwgU2lnbWEgPSB2Y292KGEpLCBUZXJtcyA9IDQ6NSkKCmBgYAoKV2FsZCB0ZXN0IGZvciBpbnRlcmFjdGlvbiBpcyBwIDwgMC4wNS4KPGJyPgo8YnI+Cjxicj4KPGJyPgpNb2RlbCBvdXRwdXQgd2l0aCBpbnRlcmFjdGlvbgpgYGB7cn0KbW0wIDwtIGNveHBoKFN1cnYoQ01UQVIsIENNKSB+IFR4KlByZXZDTSArIHN0cmF0YShQYXJpdHkpICsgc3RyYXRhKERPTVljYXQpICsgc3RyYXRhKERPU0NNKSArIGNsdXN0ZXIoRkFSTUlEKSwgZGF0YT1TRENUQ09XKQp0aWR5KG1tMCkKYGBgCkFjY29yZGluZyB0byB0aGlzIGludGVyYWN0aW9uIG1vZGVsLCB0aGUgSFIgKHJlZmVyZW50ID0gQmxhbmtldCBUeCwgbm8gcHJldiBDTSkgZm9yIGVhY2ggZ3JvdXAgYXJlOgoKQmxhbmtldDpQcmV2MCA9IDEKCkJsYW5rZXQ6UHJldjEgPSAyLjEKCkN1bHR1cmU6UHJldkNNMCA9IDAuOTMKCkN1bHR1cmU6UHJldkNNMSA9IDAuOTcKCkFsZ29yaXRobTpQcmV2Q00wID0gMC45MwoKQWxnb3JpdGhtOlByZXZDTTEgPSAxLjE0CgpUaGlzIGlzIGEgY291bnRlci1pbnR1aXRpdmUgcmVzdWx0OiBoYXphcmRzIG9mIGNsaW5pY2FsIG1hc3RpdGlzIGFyZSBoaWdoZXIgaW4gYmxhbmtldCBjb3dzIHdpdGggcHJldmlvdXMgaGlzdG9yeSBvZiBDTS4KCkhvd2V2ZXIsIHRoZXNlIGVmZmVjdCBlc3RpbWF0ZXMgaGF2ZSB3aWRlIGNvbmZpZGVuY2UgaW50ZXJ2YWxzLCBhbmQgdGhpcyBpbnRlcmFjdGlvbiBtb2RlbCBhZGRzIHVubmVjZXNzYXJ5IGNvbXBsZXhpdHkgYW5kIGRvZXMgbm90IGNoYW5nZSBvdXIgY29uY2x1c2lvbnMuIAoKRGVjaXNpb246IFJlcG9ydCBtYWluIGVmZmVjdHMuIAoKPGJyPgo8YnI+Cjxicj4KPGJyPgoKIyMjIFN0ZXAgNjogUmVtb3ZlIHVubmVjZXNzYXJ5IGNvdmFyaWF0ZXMgKGJhY2t3YXJkcyBzZWxlY3Rpb24gdXNpbmcgMTAlIHJ1bGUpCgpUaGUgb3JkZXIgZm9yIHJlbW92aW5nIGNvdmFyaWF0ZXMgd2lsbCBiZSBpbiBpbmNyZWFzaW5nIGxpa2VsaWhvb2Qgb2YgYmVpbmcgYSBjb25mb3VuZGVyLCB3aGljaCBpcyBiYXNlZCBvbiBteSBrbm93bGVkZ2UgYWJvdXQgdGhlIHZhcmlhYmxlcyBhbmQgdGhlaXIgZGlzdHJpYnV0aW9uIGluIHRyZWF0bWVudCBncm91cHMuCgpUaGlzIG9yZGVyIHdpbGwgYmU6IERPTVljYXQsIERPU0NNLCBQYXJpdHksIFByZXZDTQoKU3RlcCA2YTogRnVsbCBtb2RlbApgYGB7cn0KbW0wIDwtIHRpZHkoY294cGgoU3VydihDTVRBUiwgQ00pIH4gc3RyYXRhKFBhcml0eSkgKyBzdHJhdGEoRE9NWWNhdCkgKyBzdHJhdGEoRE9TQ00pICsgUHJldkNNICsgVHggKyBjbHVzdGVyKEZBUk1JRCksIGRhdGE9U0RDVENPVykpICU+JSBzZWxlY3QoInRlcm0iLCJlc3RpbWF0ZSIpCm1tMCRIUiA8LSBleHAobW0wJGVzdGltYXRlKQptbTAKYGBgCjxicj4KPGJyPgo8YnI+Cjxicj4KClN0ZXAgNmI6IHJlbW92aW5nIERPTVljYXQKYGBge3J9Cm1tMCA8LSB0aWR5KGNveHBoKFN1cnYoQ01UQVIsIENNKSB+IHN0cmF0YShQYXJpdHkpICsgc3RyYXRhKERPU0NNKSArIFByZXZDTSArIFR4ICsgY2x1c3RlcihGQVJNSUQpLCBkYXRhPVNEQ1RDT1cpKSU+JSBzZWxlY3QoInRlcm0iLCJlc3RpbWF0ZSIpCm1tMCRIUiA8LSBleHAobW0wJGVzdGltYXRlKQptbTAKYGBgCkJvdGggSFIgKEN1bHR1cmUgYW5kIEFsZ29yaXRobSkgY2hhbmdlZCBieSA8MTAlLiBET01ZY2F0IHN0YXlzIG91dC4gCjxicj4KPGJyPgo8YnI+Cjxicj4KU3RlcCA2YzogcmVtb3ZpbmcgRE9TQ00KYGBge3J9Cm1tMCA8LSB0aWR5KGNveHBoKFN1cnYoQ01UQVIsIENNKSB+IHN0cmF0YShQYXJpdHkpICsgUHJldkNNICsgVHggKyBjbHVzdGVyKEZBUk1JRCksIGRhdGE9U0RDVENPVykpICU+JSBzZWxlY3QoInRlcm0iLCJlc3RpbWF0ZSIpCm1tMCRIUiA8LSBleHAobW0wJGVzdGltYXRlKQptbTAKYGBgCkJvdGggSFIgY2hhbmdlZCBieSA8MTAlLiBET01ZY2F0IHN0YXlzIG91dC4gCjxicj4KPGJyPgo8YnI+Cjxicj4KClN0ZXAgNmQ6IHJlbW92aW5nIFBhcml0eQpgYGB7cn0KbW0wIDwtIHRpZHkoY294cGgoU3VydihDTVRBUiwgQ00pIH4gUHJldkNNICsgVHggKyBjbHVzdGVyKEZBUk1JRCksIGRhdGE9U0RDVENPVykpICU+JSBzZWxlY3QoInRlcm0iLCJlc3RpbWF0ZSIpCm1tMCRIUiA8LSBleHAobW0wJGVzdGltYXRlKQptbTAKYGBgCkJvdGggSFIgY2hhbmdlZCBieSA8MTAlLiBQYXJpdHkgc3RheXMgb3V0LgoKPGJyPgo8YnI+Cjxicj4KPGJyPgpTdGVwIDZlOiByZW1vdmluZyBQcmV2Q00KYGBge3J9Cm1tMCA8LSB0aWR5KGNveHBoKFN1cnYoQ01UQVIsIENNKSB+IFR4ICsgY2x1c3RlcihGQVJNSUQpLCBkYXRhPVNEQ1RDT1cpKSAlPiUgc2VsZWN0KCJ0ZXJtIiwiZXN0aW1hdGUiKQptbTAkSFIgPC0gZXhwKG1tMCRlc3RpbWF0ZSkKbW0wCmBgYApCb3RoIEhSIGNoYW5nZWQgYnkgPDEwJS4gIFByZXZDTSBzdGF5cyBvdXQuIAo8YnI+Cjxicj4KPGJyPgo8YnI+CgojIyMgU3RlcCA3YTogUmVwb3J0aW5nIGZpbmFsIGNveCBtb2RlbCBmb3IgY2xpbmljYWwgbWFzdGl0aXMgKDEtMTIwIERJTSkKTm8gY292YXJpYXRlcyBpbmNsdWRlZCwgYXMgbm8gZXZpZGVuY2UgZm9yIGNvbmZvdW5kaW5nLgpgYGB7cn0KCkNveENNIDwtIHRpZHkoY294cGgoU3VydihDTVRBUiwgQ00pIH4gVHggKyBjbHVzdGVyKEZBUk1JRCksIGRhdGE9U0RDVENPVykpCkNveENNJEhSIDwtIGV4cChDb3hDTSRlc3RpbWF0ZSkKQ294Q00kTENMIDwtIGV4cChDb3hDTSRjb25mLmxvdykKQ294Q00kVUNMIDwtIGV4cChDb3hDTSRjb25mLmhpZ2gpCkNveENNIDwtIENveENNICU+JSBzZWxlY3QodGVybSxIUixMQ0wsVUNMLHJvYnVzdC5zZSxwLnZhbHVlKQpDb3hDTQpgYGAKCiMjIyMgQ2hlY2tpbmcgU2Nob2VuZmllbGQgcmVzaWR1YWxzIGZvciBUeCBpbiBmaW5hbCBtb2RlbApgYGB7ciBmaWcuaGVpZ2h0PTh9ClNSIDwtIGNveHBoKFN1cnYoQ01UQVIsIENNKSB+IFR4ICsgY2x1c3RlcihGQVJNSUQpLCBkYXRhPVNEQ1RDT1cpCmNveC56cGgoU1IpClNSIDwtIGNveC56cGgoU1IpCmdnY294enBoKFNSLHZhciA9IGMoIlR4Q3VsdHVyZSIsIlR4QWxnb3JpdGhtIikpCmBgYApTdWZmaWNpZW50IGV2aWRlbmNlIHRvIHVzZSBDb3ggbW9kZWwuCgojIyMgU3RlcCA3YjogUmVwb3J0aW5nIGZpbmFsIGNveCBtb2RlbCBmb3IgY2xpbmljYWwgbWFzdGl0aXMgKDEtMTIwIERJTSkgdXNpbmcgUC12YWx1ZSBiYXNlZCBiYWNrd2FyZHMgc3RlcHdpc2Ugc2VsZWN0aW9uCmBgYHtyfQpDb3hDTSA8LSBjb3hwaChTdXJ2KENNVEFSLCBDTSkgfiBUeCArIFBhcml0eSArIGNsdXN0ZXIoRkFSTUlEKSwgZGF0YT1TRENUQ09XKSAlPiUgdGlkeSgpCkNveENNJEhSIDwtIGV4cChDb3hDTSRlc3RpbWF0ZSkKQ294Q00kTENMIDwtIGV4cChDb3hDTSRjb25mLmxvdykKQ294Q00kVUNMIDwtIGV4cChDb3hDTSRjb25mLmhpZ2gpCkNveENNIDwtIENveENNICU+JSBzZWxlY3QodGVybSxIUixMQ0wsVUNMLHJvYnVzdC5zZSxwLnZhbHVlKQpDb3hDTQpgYGAKRXN0aW1hdGVzIGFyZSB2ZXJ5IHNpbWlsYXIgdG8gdGhvc2UgZm91bmQgdXNpbmcgMTAlIHJ1bGUuCjxicj4KPGJyPgo8YnI+Cjxicj4KCgojIE91dGNvbWUgMjogQ3VsbGluZyAvIERlYXRoCiMjIEthcGxhbiBNZWllciBjdXJ2ZSAmIGxvZyByYW5rIHRlc3QgZm9yIGN1bGxpbmcvZGVhdGgKCkthcGxhbiBNZWllciBjdXJ2ZQpgYGB7ciBmaWcuaGVpZ2h0PTh9CktNIDwtIHN1cnZmaXQoU3VydihDdWxsMlRBUiwgQ3VsbDIpIH4gVHgsIGRhdGEgPSBTRENUQ09XKQpnZ3N1cnZwbG90KEtNLCBkYXRhID0gU0RDVENPVywgIHRpdGxlID0gIiIsIHB2YWwgPSBULCBjb25mLmludCA9IEYscmlzay50YWJsZS5jb2wgPSAiVHgiLHJpc2sudGFibGUgPSBULCByaXNrLnRhYmxlLnkudGV4dC5jb2wgPSBUUlVFICwgc3Vydi5wbG90LmhlaWdodCA9IDUsIGxlZ2VuZC5sYWJzID0gYygiQmxhbmtldCIsIkN1bHR1cmUiLCJBbGdvcml0aG0iKSwgdGFibGVzLnRoZW1lID0gdGhlbWVfY2xlYW50YWJsZSgpLCBnZ3RoZW1lID0gdGhlbWVfYncoKSkKYGBgCjxicj4KPGJyPgo8YnI+Cjxicj4KTG9nLVJhbmsgdGVzdApgYGB7cn0Kc3VydmRpZmYoU3VydihDdWxsMlRBUiwgQ3VsbDIpIH4gVHgsZGF0YT1TRENUQ09XLHJobz0wKQpgYGAKCgojIyBDb3ggcHJvcG9ydGlvbmFsIGhhemFyZHMgcmVncmVzc2lvbiBmb3IgY3VsbGluZyBvciBkZWF0aCAoMS0xMjAgRElNKQojIyBNb2RlbCBidWlsZGluZyBwbGFuCgpNb2RlbCB0eXBlOiBDb3ggcHJvcG9ydGlvbmFsIGhhemFyZHMgYW5hbHlzaXMgd2l0aCBmYXJtIGluY2x1ZGVkIGFzIGEgY2x1c3RlciB2YXJpYWJsZSB0byBhY2NvdW50IGZvciBsYWNrIG9mIGluZGVwZWRlbmNlLgoKU3RlcCAxOiBJZGVudGlmeSBwb3RlbnRpYWwgY29uZm91ZGVycyB1c2luZyBhIGRpcmVjdGVkIGFjeWNsaWMgZ3JhcGggKERBRykKClN0ZXAgMjogSWRlbnRpZnkgY29ycmVsYXRlZCB2YXJpYWJsZXMgdXNpbmcgcGVhcnNvbiBhbmQga2VuZGFsbHMgY29ycmVsYXRpb24gY29lZmZpY2llbnRzCgpTdGVwIDM6IENyZWF0ZSBtb2RlbCB3aXRoIGFsbCBwb3RlbnRpYWwgY29uZm91bmRlcnMKClN0ZXAgNDogSW52ZXN0aWdhdGUgaWYgY292YXJpYXRlcyBtZWV0IHByb3BvcnRpb25hbCBoYXphcmRzIGFzc3VtcHRpb24KClN0ZXAgNTogSW52ZXN0aWdhdGUgcG90ZW50aWFsIGVmZmVjdCBtZWFzdXJlIG1vZGlmaWNhdGlvbgoKU3RlcCA2OiBSZW1vdmUgdW5uZWNjZXNhcnkgY292YXJpYXRlcyBpbiBiYWNrd2FyZHMgc3RlcHdpc2UgZmFzaGlvbiB1c2luZyAxMCUgcnVsZSAoaS5lLiBpZiBIUiBjaGFuZ2VzIGJ5ID4xMCUgYWZ0ZXIgcmVtb3ZhbCBvZiBjb3ZhcmlhdGUsIHRoZSBjb3ZhcmlhdGUgaXMgcmV0YWluZWQgaW4gdGhlIG1vZGVsKQoKU3RlcCA3OiBSZXBvcnQgZmluYWwgbW9kZWwKCgojIyMgU3RlcHMgMSAmIDI6IElkZW50aWZ5IHBvdGVudGlhbCBjb25mb3VkZXJzIHVzaW5nIGEgZGlyZWN0ZWQgYWN5Y2xpYyBncmFwaCAoREFHKQpgYGB7ciBmaWcud2lkdGg9MTB9CmxpYnJhcnkoRGlhZ3JhbW1lUikKbWVybWFpZCgiZ3JhcGggTFIKICAgICAgICBUKFRyZWF0bWVudCktLT5VKEN1bGxpbmcgb3IgRGVhdGgpCiAgICAgICAgQShBZ2UpLS0+VAogICAgICAgIFAoUGFyaXR5KS0tPlQKICAgICAgICBNKFlpZWxkIGF0IGRyeS1vZmYpLS0+VAogICAgICAgIFMoU0NDIGR1cmluZyBwcmV2IGxhY3RhdGlvbiktLT5UCiAgICAgICAgQyhDTSBpbiBwcmV2IGxhY3QpLS0+VAogICAgICAgIEEtLT5VCiAgICAgICAgUC0tPlUKICAgICAgICBNLS0+VQogICAgICAgIFMtLT5VCiAgICAgICAgQy0tPlUKICAgICAgICBDLS0+TQogICAgICAgIFAtLT5DCiAgICAgICAgUC0tPlMKICAgICAgICBQLS0+TQogICAgICAgIEEtLT5QCiAgICAgICAgQS0tPkMKICAgICAgICBBLS0+UwogICAgICAgIEEtLT5NCiAgICAgICAgTS0tPlMKICAgICAgICBDLS0+UwogICAgICAgIHN0eWxlIEEgZmlsbDojRkZGRkZGLCBzdHJva2Utd2lkdGg6MHB4CiAgICAgICAgc3R5bGUgVCBmaWxsOiNGRkZGRkYsIHN0cm9rZS13aWR0aDoycHgKICAgICAgICBzdHlsZSBQIGZpbGw6I0ZGRkZGRiwgc3Ryb2tlLXdpZHRoOjBweAogICAgICAgIHN0eWxlIE0gZmlsbDojRkZGRkZGLCBzdHJva2Utd2lkdGg6MHB4CiAgICAgICAgc3R5bGUgUyBmaWxsOiNGRkZGRkYsIHN0cm9rZS13aWR0aDowcHgKICAgICAgICBzdHlsZSBDIGZpbGw6I0ZGRkZGRiwgc3Ryb2tlLXdpZHRoOjBweAogICAgICAgIHN0eWxlIEkgZmlsbDojRkZGRkZGLCBzdHJva2Utd2lkdGg6MHB4CiAgICAgICAgc3R5bGUgVSBmaWxsOiNGRkZGRkYsIHN0cm9rZS13aWR0aDoycHgKICAgICAgICAiKQpgYGAKQWNjb3JkaW5nIHRvIHRoaXMgREFHLCBJIG1heSBuZWVkIHRvIGNvbnRyb2wgZm9yIHRoZSBmb2xsb3dpbmcgdmFyaWFibGVzOgoKUGFyaXR5IFsiUGFyaXR5Il0uICBXaWxsIG5vdCBvZmZlciBBZ2UgWyJBZ2UiXSBhcyBpdCBpcyBoaWdobHkgY29ycmVsYXRlZC4KCllpZWxkIGF0IG1vc3QgcmVjZW50IHRlc3QgYmVmb3JlIGRyeSBvZmYgWyJET01ZIl0KClNvbWF0aWMgY2VsbCBjb3VudCBhdCBsYXN0IGhlcmQgdGVzdCBkdXJpbmcgcHJldmlvdXMgbGFjdGF0aW9uIFsiRE9TQ0MiXSA8LSB3aWxsIG5vdCBvZmZlciBQcmV2U0NDSEkgYXMgaXQgd2FzIGNvcnJlbGF0ZWQgd2l0aCBET1NDQwoKQ2xpbmljYWwgbWFzdGl0aXMgaW4gcHJldmlvdXMgbGFjdGF0aW9uIFsiUHJldkNNIl0KCjxicj4KPGJyPgo8YnI+Cjxicj4KCiMjIyBTdGVwIDM6IENyZWF0ZSBtb2RlbCB3aXRoIGFsbCBwb3RlbnRpYWwgY29uZm91bmRlcnMKCkNvd3MgdGhhdCBkaWQgbm90IGNhbHZlIGFyZSBleGNsdWRlZCAobGVmdCBjZW5zb3JlZCkKClJlYXNvbiBmb3IgUiBjZW5zb3IgPSAxMjAgRElNCgpgYGB7cn0KbGlicmFyeShicm9vbSkKbGlicmFyeShzdXJ2aXZhbCkKY294cGgoU3VydihDdWxsMlRBUiwgQ3VsbDIpIH4gUGFyaXR5ICsgRE9NWSArIERPU0NDICsgUHJldkNNICsgVHggKyBjbHVzdGVyKEZBUk1JRCksIGRhdGE9U0RDVENPVykKYGBgCjxicj4KPGJyPgo8YnI+Cjxicj4KCiMjIyBTdGVwIDQ6IEludmVzdGlnYXRlIGlmIGNvdmFyaWF0ZXMgbWVldCBwcm9wb3J0aW9uYWwgaGF6YXJkcyBhc3N1bXB0aW9uCkJvdGggRE9TQ0MgYW5kIERPTVkgZG9uJ3QgbWVldCB0aGUgYXNzdW1wdGlvbiBvZiBwcm9wb3J0aW9uYWwgaGF6YXJkcwoKCmBgYHtyfQpTUiA8LSBjb3guenBoKGNveHBoKFN1cnYoQ3VsbDJUQVIsIEN1bGwyKSB+IFBhcml0eSArIERPTVkgKyBET1NDQyArIFByZXZDTSArIFR4ICsgY2x1c3RlcihGQVJNSUQpLCBkYXRhPVNEQ1RDT1cpKQpTUgpgYGAKCgoKU2Nob2VuZmllbGQgcmVzaWR1YWwgcGxvdHMKYGBge3IgZmlnLmhlaWdodD04fQpnZ2NveHpwaChTUix2YXIgPSBjKCJUeEN1bHR1cmUiLCJUeEFsZ29yaXRobSIpKQpgYGAKPGJyPgo8YnI+Cjxicj4KPGJyPgpET1NDQyBhbmQgRE9NWSBkb24ndCBtZWV0IHRoZSBhc3N1bXB0aW9uIG9mIHByb3BvcnRpb25hbCBoYXphcmRzCmBgYHtyIGZpZy5oZWlnaHQ9OH0KZ2djb3h6cGgoU1IsdmFyID0gYygiRE9TQ0MiLCAiRE9NWSIpKQpgYGAKYGBge3IgZmlnLmhlaWdodD04fQpnZ2NveHpwaChTUix2YXIgPSBjKCJQYXJpdHkyIiwiUGFyaXR5MyIpKQpgYGAKYGBge3IgZmlnLmhlaWdodD04fQpnZ2NveHpwaChTUix2YXIgPSBjKCJQcmV2Q00xIikpCmBgYAo8YnI+Cjxicj4KPGJyPgo8YnI+CgpSZWZpdCBjb3ggbW9kZWwgd2l0aCBzZXBlcmF0ZSBiYXNlbGluZXMgKHN0cmF0aWZpZWQpIGZvciBET01ZIGFuZCBET1NDQy4KCk5vIGV2aWRlbmNlIG9mIG90aGVyIHRpbWUtdmFyeWluZyBjb3ZhcmlhdGVzIG9yIHByZWRpY3RvcnMgKFR4KS4gIApgYGB7cn0KU1IgPC0gY294cGgoU3VydihDdWxsMlRBUiwgQ3VsbDIpIH4gUGFyaXR5ICsgc3RyYXRhKERPTVljYXQpICsgc3RyYXRhKERPU0NNKSArIFByZXZDTSArIFR4ICsgY2x1c3RlcihGQVJNSUQpLCBkYXRhPVNEQ1RDT1cpClNSIDwtIGNveC56cGgoU1IpClNSCmBgYAo8YnI+Cjxicj4KPGJyPgo8YnI+CgojIyMgU3RlcCA1OiBBc3Nlc3MgZm9yIGVmZmVjdCBtZWFzdXJlIG1vZGlmaWNhdGlvbgoKVHggOiBGQVJNSUQgLSBjb3VsZCBub3QgYXNzZXNzIHRoaXMgdXNpbmcgQ294IHJlZ3Jlc3Npb24gYXMgbW9kZWwgZGlkIG5vdCBjb252ZXJnZSBvciByZXBvcnRlZCB1bnVzdWFsbHkgbGFyZ2UgU0VzIGZvciBzb21lIGludGVyYWN0aW9uIHRlcm1zLiAKCmBgYHtyfQpTUiA8LSBjb3hwaChTdXJ2KEN1bGwyVEFSLCBDdWxsMikgfiBUeCpGQVJNSUQgKyBQYXJpdHkgKyBQcmV2Q00gKyBQYXJpdHkgKyBzdHJhdGEoRE9NWWNhdCkgKyBzdHJhdGEoRE9TQ00pLCBkYXRhPVNEQ1RDT1cpCnRpZHkoU1IpCmBgYAo8YnI+Cjxicj4KPGJyPgo8YnI+Ckkgd2lsbCBhc3Nlc3MgZWZmZWN0IFR4OmZhcm0gaW50ZXJhY3Rpb24gdXNpbmcgbG9naXN0aWMgcmVncmVzc2lvbjogCgpXYWxkIHRlc3QgZm9yIGludGVyYWN0aW9uIHRlcm0gd2FzIFAgPiAwLjA1LiAKYGBge3J9Cm1tMCA8LSBnbG0oQ00gfiBUeCpGQVJNSUQgKyBQYXJpdHkgKyBQcmV2Q00gKyBQYXJpdHkgKyBET01ZY2F0ICsgRE9TQ00sIGRhdGE9U0RDVENPVykKY2FyOjpBbm92YShtbTApCmBgYAo8YnI+Cjxicj4KPGJyPgo8YnI+CkFzc2VzcyBmb3Igb3RoZXIgaW50ZXJhY3Rpb25zIHVzaW5nIFdhbGQgdGVzdGZyb20gQ294IHJlZ3Jlc3Npb24uIAoKV2FsZCB0ZXN0IHdhcyBQID4gMC4wNSBmb3IgVHg6UHJldkNNIGludGVyYWN0aW9uCi4gCmBgYHtyfQptbTAgPC0gY294cGgoU3VydihDdWxsMlRBUiwgQ3VsbDIpIH4gVHgqUHJldkNNICsgUGFyaXR5ICsgc3RyYXRhKERPTVljYXQpICsgc3RyYXRhKERPU0NNKSArIGNsdXN0ZXIoRkFSTUlEKSwgZGF0YT1TRENUQ09XKQpzdW1tYXJ5KG1tMCkKd2FsZC50ZXN0KGIgPSBjb2VmKG1tMCksIFNpZ21hID0gdmNvdihtbTApLCBUZXJtcyA9IDY6NykKYGBgCjxicj4KPGJyPgo8YnI+Cjxicj4KV2FsZCB0ZXN0IHdhcyBQIDwgMC4wNSBmb3IgVHg6UGFyaXR5IGludGVyYWN0aW9uCi4gCmBgYHtyfQptbTAgPC0gY294cGgoU3VydihDdWxsMlRBUiwgQ3VsbDIpIH4gVHgqUGFyaXR5ICsgUHJldkNNICsgc3RyYXRhKERPTVljYXQpICsgc3RyYXRhKERPU0NNKSArIGNsdXN0ZXIoRkFSTUlEKSwgZGF0YT1TRENUQ09XKQpzdW1tYXJ5KG1tMCkKd2FsZC50ZXN0KGIgPSBjb2VmKG1tMCksIFNpZ21hID0gdmNvdihtbTApLCBUZXJtcyA9IDY6OSkKYGBgCjxicj4gCjxicj4KPGJyPgpJbnZlc3RpZ2F0ZSBwb3RlbnRpYWwgZWZmZWN0IG1lYXN1cmUgbW9kaWZpY2F0aW9uIGJ5IGRvaW5nIHN0cmF0aWZpZWQgbW9kZWxzCgpQYXJpdHkgPSAxCkZyZXF1ZW5jeSB0YWJsZQpgYGB7cn0KdGFibGUoU0RDVENPVyRUeFtTRENUQ09XJFBhcml0eT09MV0sU0RDVENPVyRDdWxsMltTRENUQ09XJFBhcml0eT09MV0pCmBgYAo8YnI+Cjxicj4KPGJyPgoKIyMjIyBNb2RlbCAoUGFyaXR5ID0gMSkKYGBge3J9Cm1tMCA8LSBjb3hwaChTdXJ2KEN1bGwyVEFSLCBDdWxsMikgfiBUeCArIFByZXZDTSArIHN0cmF0YShET01ZY2F0KSArIHN0cmF0YShET1NDTSkgKyBjbHVzdGVyKEZBUk1JRCksIGRhdGE9U0RDVENPV1tTRENUQ09XJFBhcml0eT09MSxdKSAlPiUgdGlkeSgpCm1tMCRIUiA8LSBleHAobW0wJGVzdGltYXRlKQptbTAkTENMIDwtIGV4cChtbTAkY29uZi5sb3cpCm1tMCRVQ0wgPC0gZXhwKG1tMCRjb25mLmhpZ2gpCm1tMCAlPiUgc2VsZWN0KCJ0ZXJtIiwiSFIiLCJMQ0wiLCJVQ0wiKQpgYGAKCjxicj4KPGJyPgo8YnI+CgojIyMjIFBhcml0eSA9IDIKCkZyZXF1ZW5jeSB0YWJsZQpgYGB7cn0KdGFibGUoU0RDVENPVyRUeFtTRENUQ09XJFBhcml0eT09Ml0sU0RDVENPVyRDdWxsMltTRENUQ09XJFBhcml0eT09Ml0pCmBgYAoKYGBge3J9Cm1tMCA8LSBjb3hwaChTdXJ2KEN1bGwyVEFSLCBDdWxsMikgfiBUeCArIFByZXZDTSArIHN0cmF0YShET01ZY2F0KSArIHN0cmF0YShET1NDTSkgKyBjbHVzdGVyKEZBUk1JRCksIGRhdGE9U0RDVENPV1tTRENUQ09XJFBhcml0eT09MixdKSAlPiUgdGlkeSgpCm1tMCRIUiA8LSBleHAobW0wJGVzdGltYXRlKQptbTAkTENMIDwtIGV4cChtbTAkY29uZi5sb3cpCm1tMCRVQ0wgPC0gZXhwKG1tMCRjb25mLmhpZ2gpCm1tMCAlPiUgc2VsZWN0KCJ0ZXJtIiwiSFIiLCJMQ0wiLCJVQ0wiKQpgYGAKCjxicj4KPGJyPgo8YnI+CgojIyMjIFBhcml0eSA9IDMgb3IgZ3JlYXRlcgpGcmVxdWVuY3kgdGFibGUKYGBge3J9CnRhYmxlKFNEQ1RDT1ckVHhbU0RDVENPVyRQYXJpdHk9PTNdLFNEQ1RDT1ckQ3VsbDJbU0RDVENPVyRQYXJpdHk9PTNdKQpgYGAKCmBgYHtyfQptbTAgPC0gY294cGgoU3VydihDdWxsMlRBUiwgQ3VsbDIpIH4gVHggKyBQcmV2Q00gKyBzdHJhdGEoRE9NWWNhdCkgKyBzdHJhdGEoRE9TQ00pICsgY2x1c3RlcihGQVJNSUQpLCBkYXRhPVNEQ1RDT1dbU0RDVENPVyRQYXJpdHk9PTMsXSkgJT4lIHRpZHkKbW0wJEhSIDwtIGV4cChtbTAkZXN0aW1hdGUpCm1tMCRMQ0wgPC0gZXhwKG1tMCRjb25mLmxvdykKbW0wJFVDTCA8LSBleHAobW0wJGNvbmYuaGlnaCkKbW0wICU+JSBzZWxlY3QoInRlcm0iLCJIUiIsIkxDTCIsIlVDTCIpCmBgYAoKVGhpcyBmaW5kaW5nIHN1Z2dlc3RzIHRoYXQgcG90ZW50aWFsIGN1bGxpbmcgcmlza3MgYXNzb2NpYXRlZCB3aXRoIFNEQ1QgbWF5IGJlIGhpZ2hlciBpbiBwYXJpdHkgMSBjb3dzIHRoYW4gaW4gbXVsdGlwYXJvdXMgYW5pbWFscy4gIEhvd2V2ZXIgdGhlc2UgSFIgZXN0aW1hdGVzIGluIGFsbCAzIHBhcml0eSBncm91cHMgYXJlIGltcHJlY2lzZSAod2lkZSBjb25maWRlbmNlIGludGVydmFscykgYmVjYXVzZSBvZiBhIHNtYWxsIG51bWJlciBvZiBjdWxsaW5nL2RlYXRoIGV2ZW50cy4gIEZvciBleGFtcGxlLCBpbiBsYWN0YXRpb24gMSBhbmltYWxzOiA2IGluIEJEQ1QsIDExIGluIEFsZ29yaXRobSBhbmQgMTIgaW4gQ3VsdHVyZS4gVGhlcmVmb3JlLCB0aGlzIGRpZmZlcmVuY2UgaW4gZWZmZWN0cyBhY3Jvc3MgbGV2ZWxzIG9mIHBhcml0eSBtYXkgYmUgZHVlIHRvIGNoYW5jZSBhbG9uZSwgYW5kIG5vdCB3b3J0aCBpbnZlc3RpZ2F0aW5nIHRvbyBjbG9zZWx5LiAKCkRlY2lzaW9uOiBSZXBvcnQgbWFpbiBlZmZlY3RzLgoKPGJyPgo8YnI+Cjxicj4KCiMjIyBTdGVwIDY6IFJlbW92ZSB1bm5lY2Vzc2FyeSBjb3ZhcmlhdGVzIChiYWNrd2FyZHMgc2VsZWN0aW9uIHVzaW5nIDEwJSBydWxlKQoKVGhlIG9yZGVyIGZvciByZW1vdmluZyBjb3ZhcmlhdGVzIHdpbGwgYmUgaW4gaW5jcmVhc2luZyBsaWtlbGlob29kIG9mIGJlaW5nIGEgY29uZm91bmRlciwgd2hpY2ggaXMgYmFzZWQgb24gbXkga25vd2xlZGdlIGFib3V0IHRoZSB2YXJpYWJsZXMgYW5kIHRoZWlyIGRpc3RyaWJ1dGlvbiBpbiB0cmVhdG1lbnQgZ3JvdXBzLgoKVGhpcyBvcmRlciB3aWxsIGJlOiBET01ZY2F0LCBET1NDTSwgUHJldkNNLCBQYXJpdHkKClN0ZXAgNmE6IEZ1bGwgbW9kZWwKYGBge3J9Cm1tMCA8LSBjb3hwaChTdXJ2KEN1bGwyVEFSLCBDdWxsMikgfiBUeCArIFByZXZDTSArIFBhcml0eSArIHN0cmF0YShET01ZY2F0KSArIHN0cmF0YShET1NDTSkgKyBjbHVzdGVyKEZBUk1JRCksIGRhdGE9U0RDVENPVykgJT4lIHRpZHkoKSAlPiUgc2VsZWN0KCJ0ZXJtIiwiZXN0aW1hdGUiKQptbTAkSFIgPC0gZXhwKG1tMCRlc3RpbWF0ZSkKbW0wCmBgYAo8YnI+Cjxicj4KPGJyPgo8YnI+ClN0ZXAgNmI6IHJlbW92aW5nIERPTVljYXQKYGBge3J9Cm1tMCA8LSBjb3hwaChTdXJ2KEN1bGwyVEFSLCBDdWxsMikgfiBUeCArIFByZXZDTSArIFBhcml0eSArIHN0cmF0YShET1NDTSkgKyBjbHVzdGVyKEZBUk1JRCksIGRhdGE9U0RDVENPVykgJT4lIHRpZHkoKSAlPiUgc2VsZWN0KCJ0ZXJtIiwiZXN0aW1hdGUiKQptbTAkSFIgPC0gZXhwKG1tMCRlc3RpbWF0ZSkKbW0wCmBgYApIUiBmb3IgY3VsdHVyZSBhbmQgYWxnb3JpdGhtIGRpZG4ndCBjaGFuZ2UgYnkgPjEwJS4gIERPTVljYXQgc3RheXMgb3V0LiAKPGJyPgo8YnI+Cjxicj4KPGJyPgoKU3RlcCA2YzogcmVtb3ZpbmcgRE9TQ00KYGBge3J9Cm1tMCA8LSBjb3hwaChTdXJ2KEN1bGwyVEFSLCBDdWxsMikgfiBUeCArIFByZXZDTSArIFBhcml0eSArIGNsdXN0ZXIoRkFSTUlEKSwgZGF0YT1TRENUQ09XKSAlPiUgdGlkeSgpICU+JSBzZWxlY3QoInRlcm0iLCJlc3RpbWF0ZSIpCm1tMCRIUiA8LSBleHAobW0wJGVzdGltYXRlKQptbTAKYGBgCkhSIGNoYW5nZWQgYnkgPDEwJS4gIERPU0NNIHN0YXlzIG91dC4gCjxicj4KPGJyPgo8YnI+Cjxicj4KClN0ZXAgNmQ6IHJlbW92aW5nIFByZXZDTQpgYGB7cn0KbW0wIDwtIGNveHBoKFN1cnYoQ3VsbDJUQVIsIEN1bGwyKSB+IFR4ICsgUGFyaXR5ICsgY2x1c3RlcihGQVJNSUQpLCBkYXRhPVNEQ1RDT1cpICU+JSB0aWR5KCkgJT4lIHNlbGVjdCgidGVybSIsImVzdGltYXRlIikKbW0wJEhSIDwtIGV4cChtbTAkZXN0aW1hdGUpCm1tMApgYGAKSFIgY2hhbmdlZCBieSA8MTAlLiAgTGVhdmUgUHJldkNNIG91dC4gCjxicj4KPGJyPgo8YnI+Cjxicj4KU3RlcCA2ZTogcmVtb3ZpbmcgUGFyaXR5CmBgYHtyfQptbTAgPC0gY294cGgoU3VydihDdWxsMlRBUiwgQ3VsbDIpIH4gVHggKyBjbHVzdGVyKEZBUk1JRCksIGRhdGE9U0RDVENPVykgJT4lIHRpZHkoKSAlPiUgc2VsZWN0KCJ0ZXJtIiwiZXN0aW1hdGUiKQptbTAkSFIgPC0gZXhwKG1tMCRlc3RpbWF0ZSkKbW0wCmBgYApIUiBjaGFuZ2VkIGJ5IDwxMCUuIFBhcml0eSBzdGF5cyBvdXQuCgo8YnI+Cjxicj4KPGJyPgo8YnI+CgojIyMgU3RlcCA3YTogRmluYWwgY294IG1vZGVsIGZvciBjdWxsaW5nIG9yIGRlYXRoICgxIC0gMTIwIERJTSkKTm8gY292YXJpYXRlcyBpbmNsdWRlZCwgYXMgbm8gZXZpZGVuY2UgZm9yIGNvbmZvdW5kaW5nLgpgYGB7cn0KQ294Q3VsbCA8LSBjb3hwaChTdXJ2KEN1bGwyVEFSLCBDdWxsMikgfiBUeCArIGNsdXN0ZXIoRkFSTUlEKSwgZGF0YT1TRENUQ09XKSAlPiUgdGlkeSgpCkNveEN1bGwkSFIgPC0gZXhwKENveEN1bGwkZXN0aW1hdGUpCkNveEN1bGwkTENMIDwtIGV4cChDb3hDdWxsJGNvbmYubG93KQpDb3hDdWxsJFVDTCA8LSBleHAoQ294Q3VsbCRjb25mLmhpZ2gpCkNveEN1bGwgPC0gQ294Q3VsbCAlPiUgc2VsZWN0KHRlcm0sSFIsTENMLFVDTCxyb2J1c3Quc2UscC52YWx1ZSkKQ294Q3VsbApgYGAKCgojIyMjIENoZWNrIHNjaG9lbmZpZWxkIHJlc2lkdWFscyBmb3IgZmluYWwgbW9kZWwKYGBge3IgZmlnLmhlaWdodD04fQpTUiA8LSBjb3hwaChTdXJ2KEN1bGwyVEFSLCBDdWxsMikgfiBUeCArIGNsdXN0ZXIoRkFSTUlEKSwgZGF0YT1TRENUQ09XKQpjb3guenBoKFNSKQpTUiA8LSBjb3guenBoKFNSKQpnZ2NveHpwaChTUix2YXIgPSBjKCJUeEN1bHR1cmUiLCJUeEFsZ29yaXRobSIpKQpgYGAKCgojIyMgU3RlcCA3YjogRmluYWwgY294IG1vZGVsIGZvciBjdWxsaW5nIG9yIGRlYXRoICgxIC0gMTIwIERJTSkgdXNpbmcgUC12YWx1ZSBiYXNlZCBiYWNrd2FyZHMgc3RlcHdpc2Ugc2VsZWN0aW9uCmBgYHtyfQpDb3hDdWxsIDwtIGNveHBoKFN1cnYoQ3VsbFRBUiwgQ3VsbDIpIH4gVHggKyBQYXJpdHkgKyBET1NDQyArIFByZXZDTSArIGNsdXN0ZXIoRkFSTUlEKSwgZGF0YT1TRENUQ09XKSAlPiUgdGlkeSgpCkNveEN1bGwkSFIgPC0gZXhwKENveEN1bGwkZXN0aW1hdGUpCkNveEN1bGwkTENMIDwtIGV4cChDb3hDdWxsJGNvbmYubG93KQpDb3hDdWxsJFVDTCA8LSBleHAoQ294Q3VsbCRjb25mLmhpZ2gpCkNveEN1bGwgPC0gQ294Q3VsbCAlPiUgc2VsZWN0KHRlcm0sSFIsTENMLFVDTCxyb2J1c3Quc2UscC52YWx1ZSkKQ294Q3VsbApgYGAKRXN0aW1hdGVzIGFyZSBzaW1pbGFyIHRvIDEwJSBydWxlIGJhc2VkIGFwcHJvYWNoCjxicj4KPGJyPgo8YnI+Cjxicj4KCgoKCgojIENoYW5naW5nIGRhdGFzZXRzOiBESEkgZGF0YXNldCB3aXRoIG11bHRpcGxlIHJvd3MgcGVyIGNvdwoKRGF0YXNldCBleGFtcGxlCmBgYHtyfQpTRENUQ09XREhJJFR4IDwtIAogIGZhY3RvcihTRENUQ09XREhJJFR4LCAKICAgICAgICAgbGV2ZWxzPWMoMCwxLDIpLAogICAgICAgICBsYWJlbHM9YygiQmxhbmtldCIsCiAgICAgICAgICAgICAgICAgICJDdWx0dXJlIiwgCiAgICAgICAgICAgICAgICAgICJBbGdvcml0aG0iKSkKCkRISWNoZWNrIDwtIFNEQ1RDT1dESEkgJT4lIHNlbGVjdChUeCxDb3dJRCxGQVJNSUQsRElNLFRlc3RESU1jYXQyMCxMU0NDLE1ZLFBhcml0eSxQcmV2Q00sRE9NWSxET1NDQykKaGVhZChESEljaGVjaywgbj0xMCkKYGBgCjxicj4KPGJyPgo8YnI+Cjxicj4KCiMgSW5zcGVjdCBkYXRhCmBgYHtyfQpESElvdXRjb21lIDwtIFNEQ1RDT1dESEkgJT4lIHNlbGVjdChTQ0MsTVksU0NNKQpESElvdXRjb21lJGxvZ1NDQyA8LSBsb2coREhJb3V0Y29tZSRTQ0MgKyAxKQojc3VtbWFyeXRvb2xzOjpkZlN1bW1hcnkoREhJb3V0Y29tZSwgc3R5bGU9J2dyaWQnKQpwcmludChzdW1tYXJ5dG9vbHM6OmRmU3VtbWFyeShESElvdXRjb21lLCB2YWxpZC5jb2w9RkFMU0UsIGdyYXBoLm1hZ25pZj0wLjgsIHN0eWxlPSJncmlkIiksIG1ldGhvZCA9ICJyZW5kZXIiKQoKYGBgClZlcnkgbGl0dGxlIG1pc3NpbmcgZGF0YS4gV2lsbCBkbyBjb21wbGV0ZSBjYXNlIGFuYWx5c2lzLgo8YnI+Cjxicj4KPGJyPgo8YnI+CgojIyBBc3Nlc3Npbmcgbm9ybWFsaXR5IGZvciBjb250aW51b3VzIG91dGNvbWUgdmFyaWFibGVzCgpTb21hdGljIGNlbGwgY291bnQgKGxvZyB4IDEwLDAwMCBjZWxscykKYGBge3J9CnFxUGxvdChsb2coU0RDVENPV0RISSRTQ0MrMSkpCmBgYAoKYGBge3J9Cmhpc3QobG9nKFNEQ1RDT1dESEkkU0NDKzEpKQpgYGAKPGJyPgo8YnI+Cjxicj4KPGJyPgpNaWxrIHlpZWxkIChrZykKYGBge3J9CnFxUGxvdChTRENUQ09XREhJJE1ZKQpgYGAKYGBge3J9Cmhpc3QoU0RDVENPV0RISSRNWSkKYGBgCjxicj4KPGJyPgo8YnI+Cjxicj4KCiMgT3V0Y29tZSAzOiBTb21hdGljIGNlbGwgY291bnQKCiMjIE1vZGVsbGluZyBwbGFuCk1vZGVsIHR5cGU6IExpbmVhciBtaXhlZCBtb2RlbHMsIHJhbmRvbSBpbnRlcmNlcHRzIGZvciBmYXJtIGFuZCBjb3cgd2lsbCBiZSBmaXR0ZWQgdG8gYWNjb3VudCBmb3IgcmVwZWF0ZWQgbWVhc3VyZXMgd2l0aGluIGNvd3MsIGFuZCBjbHVzdGVyaW5nIG9mIGNvd3Mgd2l0aGluIGhlcmRzCgpTdGVwIDE6IElkZW50aWZ5IHBvdGVudGlhbCBjb25mb3VkZXJzIHVzaW5nIGEgZGlyZWN0ZWQgYWN5Y2xpYyBncmFwaCAoREFHKQoKU3RlcCAyOiBJZGVudGlmeSBjb3JyZWxhdGVkIHZhcmlhYmxlcyB1c2luZyBwZWFyc29uIGFuZCBrZW5kYWxscyBjb3JyZWxhdGlvbiBjb2VmZmljaWVudHMKClN0ZXAgMzogQ3JlYXRlIG1vZGVsIHdpdGggYWxsIHBvdGVudGlhbCBjb25mb3VuZGVycwoKU3RlcCA0OiBJbnZlc3RpZ2F0ZSBwb3RlbnRpYWwgZWZmZWN0IG1lYXN1cmUgbW9kaWZpY2F0aW9uCgpTdGVwIDU6IFJlbW92ZSB1bm5lY2Nlc2FyeSBjb3ZhcmlhdGVzIGluIGJhY2t3YXJkcyBzdGVwd2lzZSBmYXNoaW9uIHVzaW5nIDEwJSBydWxlIChpLmUuIGlmIGVzdGltYXRlZCBkaWZmZXJlbmNlIGluIFNDQyBjaGFuZ2VzIGJ5ID4xMCUgYWZ0ZXIgcmVtb3ZpbmcgdGhlIGNvdmFyaWF0ZSwgdGhlIGNvdmFyaWF0ZSBpcyByZXRhaW5lZCBpbiB0aGUgbW9kZWwpCgpTdGVwIDY6IFJlcG9ydCBmaW5hbCBtb2RlbAoKU3RlcCA3OiBNb2RlbCBkaWFnbm9zdGljcwoKIyMjIFN0ZXAgMSAmIDIgSWRlbnRpZnkgcG90ZW50aWFsIGNvbmZvdW5kZXJzIHVzaW5nIGEgREFHCmBgYHtyIGZpZy53aWR0aD0xMH0KbGlicmFyeShEaWFncmFtbWVSKQptZXJtYWlkKCJncmFwaCBMUgogICAgICAgIFQoVHJlYXRtZW50KS0tPlUoU0NDKQogICAgICAgIEEoQWdlKS0tPlQKICAgICAgICBQKFBhcml0eSktLT5UCiAgICAgICAgTShZaWVsZCBhdCBkcnktb2ZmKS0tPlQKICAgICAgICBTKFNDQyBkdXJpbmcgcHJldiBsYWN0YXRpb24pLS0+VAogICAgICAgIEMoQ00gaW4gcHJldiBsYWN0KS0tPlQKICAgICAgICBEKERheXMgaW4gbWlsayBhdCB0ZXN0KS0tPlUKICAgICAgICBELS0+VAogICAgICAgIEEtLT5VCiAgICAgICAgUC0tPlUKICAgICAgICBNLS0+VQogICAgICAgIFMtLT5VCiAgICAgICAgQy0tPlUKICAgICAgICBDLS0+TQogICAgICAgIFAtLT5DCiAgICAgICAgUC0tPlMKICAgICAgICBQLS0+TQogICAgICAgIEEtLT5QCiAgICAgICAgQS0tPkMKICAgICAgICBBLS0+UwogICAgICAgIEEtLT5NCiAgICAgICAgTS0tPlMKICAgICAgICBDLS0+UwogICAgICAgIHN0eWxlIEEgZmlsbDojRkZGRkZGLCBzdHJva2Utd2lkdGg6MHB4CiAgICAgICAgc3R5bGUgVCBmaWxsOiNGRkZGRkYsIHN0cm9rZS13aWR0aDoycHgKICAgICAgICBzdHlsZSBQIGZpbGw6I0ZGRkZGRiwgc3Ryb2tlLXdpZHRoOjBweAogICAgICAgIHN0eWxlIE0gZmlsbDojRkZGRkZGLCBzdHJva2Utd2lkdGg6MHB4CiAgICAgICAgc3R5bGUgUyBmaWxsOiNGRkZGRkYsIHN0cm9rZS13aWR0aDowcHgKICAgICAgICBzdHlsZSBDIGZpbGw6I0ZGRkZGRiwgc3Ryb2tlLXdpZHRoOjBweAogICAgICAgIHN0eWxlIEkgZmlsbDojRkZGRkZGLCBzdHJva2Utd2lkdGg6MHB4CiAgICAgICAgc3R5bGUgRCBmaWxsOiNGRkZGRkYsIHN0cm9rZS13aWR0aDowcHgKICAgICAgICBzdHlsZSBVIGZpbGw6I0ZGRkZGRiwgc3Ryb2tlLXdpZHRoOjJweAogICAgICAgICIpCmBgYApBY2NvcmRpbmcgdG8gdGhpcyBEQUcsIEkgbWF5IG5lZWQgdG8gY29udHJvbCBmb3IgdGhlIGZvbGxvd2luZyB2YXJpYWJsZXM6CgpQYXJpdHkgWyJQYXJpdHkiXSA8LSBBZ2Ugbm90IG9mZmVyZWQgYXMgaGlnaGx5IGNvcnJlbGF0ZWQKCllpZWxkIGF0IG1vc3QgcmVjZW50IHRlc3QgYmVmb3JlIGRyeSBvZmYgWyJET01ZIl0KClNvbWF0aWMgY2VsbCBjb3VudCBhdCBsYXN0IGhlcmQgdGVzdCBkdXJpbmcgcHJldmlvdXMgbGFjdGF0aW9uIFsiRE9TQ0MiXSA8LSBQcmV2U0NDSEkgbm90IG9mZmVyZWQgYXMgY29ycmVsYXRlZAoKQ2xpbmljYWwgbWFzdGl0aXMgaW4gcHJldmlvdXMgbGFjdGF0aW9uIFsiUHJldkNNIl0KCkRheXMgaW4gbWlsayBhdCBoZXJkIHRlc3QgKGNhdGVnb3J5LCAwLTIwICIxMCIsIDIxLTQwICIzMCIgZXRjKSBbIlRlc3RESU1jYXQyMCJdCgoKCjxicj4KPGJyPgo8YnI+Cjxicj4KCiMjIyBTdGVwIDM6IENyZWF0ZSBtb2RlbCB3aXRoIGFsbCBwb3RlbnRpYWwgY29uZm91bmRlcnMKYGBge3J9Cm1tMCA8LSBsbWVyKExTQ0MgfiBUeCArIFRlc3RESU1jYXQyMCArIFBhcml0eSArIERPU0NDICsgRE9NWSArIFByZXZDTSArICgxfEZBUk1JRC9Db3dJRCksIGRhdGE9U0RDVENPV0RISSkKc3VtbWFyeShtbTApCmNhcjo6dmlmKG1tMCkKYGBgCjxicj4KPGJyPgo8YnI+Cjxicj4KCiMjIyBTdGVwIDQ6IEludmVzdGlnYXRlIGVmZmVjdCBtZWFzdXJlIG1vZGlmaWNhdGlvbgoKVHg6RkFSTSAoUCA+IDAuMDUpCmBgYHtyfQptbTAgPC0gbG1lcihMU0NDIH4gVHgqRkFSTUlEICsgUGFyaXR5ICsgVGVzdERJTWNhdDIwICsgRE9TQ0MgKyBET01ZICsgUHJldkNNICsgKDF8Q293SUQpLCBkYXRhPVNEQ1RDT1dESEkpCmNhcjo6QW5vdmEobW0wKQpgYGAKPGJyPgo8YnI+Cjxicj4KPGJyPgoKVHg6RElNIGNhdGVnb3J5IGF0IGhlcmQgdGVzdCAoUCA+IDAuMDUpCmBgYHtyfQptbTAgPC0gbG1lcihMU0NDIH4gVHgqVGVzdERJTWNhdDIwICsgUGFyaXR5ICsgRE9TQ0MgKyBET01ZICsgUHJldkNNICsgKDF8RkFSTUlEL0Nvd0lEKSwgZGF0YT1TRENUQ09XREhJKQpjYXI6OkFub3ZhKG1tMCkKYGBgCjxicj4KPGJyPgo8YnI+Cjxicj4KVHg6UGFyaXR5IChQID4gMC4wNSkKYGBge3J9Cm1tMCA8LSBsbWVyKExTQ0MgfiBUeCpQYXJpdHkgKyBUZXN0RElNY2F0MjAgKyBET1NDQyArIERPTVkgKyBQcmV2Q00gKyAoMXxGQVJNSUQvQ293SUQpLCBkYXRhPVNEQ1RDT1dESEkpCmNhcjo6QW5vdmEobW0wKQpgYGAKPGJyPgo8YnI+Cjxicj4KPGJyPgpUeDpQcmV2Q00gKFAgPiAwLjA1KQpgYGB7cn0KbW0wIDwtIGxtZXIoTFNDQyB+IFR4KlByZXZDTSArIFBhcml0eSArIFRlc3RESU1jYXQyMCArIERPU0NDICsgRE9NWSArICgxfEZBUk1JRC9Db3dJRCksIGRhdGE9U0RDVENPV0RISSkKY2FyOjpBbm92YShtbTApCmBgYAo8YnI+Cjxicj4KPGJyPgo8YnI+ClR4OkRPTVkgKFAgPiAwLjA1KQpgYGB7cn0KbW0wIDwtIGxtZXIoTFNDQyB+IFR4KkRPTVkgKyBQcmV2Q00gKyBQYXJpdHkgKyBUZXN0RElNY2F0MjAgKyBET1NDQyArICgxfEZBUk1JRC9Db3dJRCksIGRhdGE9U0RDVENPV0RISSkKY2FyOjpBbm92YShtbTApCmBgYAoKIyMjIFN0ZXAgNTogUmVtb3ZlIHVubmVjY2VzYXJ5IGNvdmFyaWF0ZXMgaW4gYmFja3dhcmRzIHN0ZXB3aXNlIGZhc2hpb24gdXNpbmcgMTAlIHJ1bGUKClRoZSBvcmRlciBmb3IgcmVtb3ZpbmcgY292YXJpYXRlcyB3aWxsIGJlIGluIGluY3JlYXNpbmcgbGlrZWxpaG9vZCBvZiBiZWluZyBhIGNvbmZvdW5kZXIsIHdoaWNoIGlzIGJhc2VkIG9uIG15IGtub3dsZWRnZSBhYm91dCB0aGUgdmFyaWFibGVzIGFuZCB0aGVpciBkaXN0cmlidXRpb24gaW4gdHJlYXRtZW50IGdyb3Vwcy4KClRoaXMgb3JkZXIgd2lsbCBiZTogRE9NWSwgUGFyaXR5LCBQcmV2Q00sIERPU0NDCgpUZXN0RElNY2F0MjAgd2lsbCBub3QgYmUgcmVtb3ZlZC4KClN0ZXAgNWE6IEZ1bGwgbW9kZWwKYGBge3J9Cm1tMCA8LSBsbWVyKExTQ0MgfiBUeCArIFByZXZDTSArIFBhcml0eSArIFRlc3RESU1jYXQyMCArIERPU0NDICsgRE9NWSArICgxfEZBUk1JRC9Db3dJRCksIGRhdGE9U0RDVENPV0RISSkKbW0wICU+JSB0aWR5KCkKYGBgCjxicj4KPGJyPgo8YnI+Cjxicj4KU3RlcCA1YjogcmVtb3ZpbmcgRE9NWQpgYGB7cn0KbW0wIDwtIGxtZXIoTFNDQyB+IFR4ICsgUHJldkNNICsgUGFyaXR5ICsgVGVzdERJTWNhdDIwICsgRE9TQ0MgKyAoMXxGQVJNSUQvQ293SUQpLCBkYXRhPVNEQ1RDT1dESEkpIAptbTAlPiUgdGlkeSgpCmBgYApDaGFuZ2VkIGJ5IDwxMCUuICBET01ZIHN0YXlzIG91dC4gCjxicj4KPGJyPgo8YnI+Cjxicj4KU3RlcCA1YjogcmVtb3ZpbmcgUGFyaXR5CmBgYHtyfQptbTAgPC0gbG1lcihMU0NDIH4gVHggKyBQcmV2Q00gKyBUZXN0RElNY2F0MjAgKyBET1NDQyArICgxfEZBUk1JRC9Db3dJRCksIGRhdGE9U0RDVENPV0RISSkKbW0wICU+JSB0aWR5KCkKYGBgCkNoYW5nZWQgYnkgPDEwJS4gUGFyaXR5IHN0YXlzIG91dC4gCgo8YnI+Cjxicj4KPGJyPgo8YnI+ClN0ZXAgNWM6IHJlbW92aW5nIFByZXZDTQpgYGB7cn0KbW0wIDwtIGxtZXIoTFNDQyB+IFR4ICsgVGVzdERJTWNhdDIwICsgRE9TQ0MgKyAoMXxGQVJNSUQvQ293SUQpLCBkYXRhPVNEQ1RDT1dESEkpIAptbTAgJT4lIHRpZHkoKQpgYGAKQ2hhbmdlZCBieSA+MTAlLiBQcmV2Q00gc3RheXMgaW4uIAo8YnI+Cjxicj4KPGJyPgo8YnI+ClN0ZXAgNWQ6IHJlbW92aW5nIERPU0NDCmBgYHtyfQptbTAgPC0gbG1lcihMU0NDIH4gVHggKyBQcmV2Q00gKyBUZXN0RElNY2F0MjAgKyAoMXxGQVJNSUQvQ293SUQpLCBkYXRhPVNEQ1RDT1dESEkpCm1tMCAlPiUgdGlkeSgpCmBgYApDaGFuZ2VkIGJ5ID4gMTAlLiAgRE9TQ0Mgc3RheXMgaW4uIAo8YnI+Cjxicj4KPGJyPgo8YnI+CgojIyMgU3RlcCA2YTogRmluYWwgbW9kZWwgZm9yIFNDQyAoMSAtIDEyMCBESU0pCmBgYHtyfQptbTAgPC0gbG1lcihMU0NDIH4gVHggKyBQcmV2Q00gKyBET1NDQyArIFRlc3RESU1jYXQyMCArICgxfEZBUk1JRC9Db3dJRCksIGRhdGE9U0RDVENPV0RISSkKbW0wICU+JSB0aWR5KGNvbmYuaW50PVRSVUUpCmBgYAoKIyMjIyBJQ0MgZm9yIFNDQyAoMSAtIDEyMCBESU0pCmBgYHtyfQptbTAgPC0gbG1lcihMU0NDIH4gMSArICgxfEZBUk1JRC9Db3dJRCksIGRhdGE9U0RDVENPV0RISSkKc3VtbWFyeShtbTApCmBgYAoKSUNDIChDb3dJRCkgPSAwLjM1CklDQyAoRkFSTUlEKSA9IDAuMDIKCk1vc3QgY2x1c3RlcmluZyBpcyBoYXBwZW5pbmcgd2l0aGluIGNvdywgd2hpY2ggaXMgbm90IGEgc3VwcmlzZSBnaXZlbiB0aGlzIGlzIGxvbmdpdHVkaW5hbCBkYXRhLiAKCiMjIyBTdGVwIDZiOiBGaW5hbCBtb2RlbCBmb3IgU0NDICgxIC0gMTIwIERJTSkgdXNpbmcgUC12YWx1ZSBiYXNlZCBiYWNrd2FyZHMgc2VsZWN0aW9uCmBgYHtyfQpsbWVyKExTQ0MgfiBUeCArIFBhcml0eSArIFByZXZDTSArIERPU0NDICsgKDF8RkFSTUlEL0Nvd0lEKSwgZGF0YT1TRENUQ09XREhJKSAlPiUgc3VtbWFyeSgpCgpsbWVyKExTQ0MgfiBUeCArIFBhcml0eSArIFByZXZDTSArIERPU0NDICsgKDF8RkFSTUlEL0Nvd0lEKSwgZGF0YT1TRENUQ09XREhJKSAlPiUgY2FyOjpBbm92YSgpICU+JSB0aWR5KCkKCmBgYApFc3RpbWF0ZXMgYXJlIHZlcnkgc2ltaWxhciB0byAxMCUgcnVsZSBiYXNlZCBhcHByb2FjaC4KPGJyPgo8YnI+Cjxicj4KPGJyPgoKIyMjIFN0ZXAgNmM6IEZpbmFsIG1vZGVsICh1c2luZyAxMCUgcnVsZSkgcmVwb3J0ZWQgYXMgZXN0aW1hdGVkIG1hcmdpbmFsIG1lYW5zICh+TFNtZWFucykKR2VvbWV0cmljIG1lYW4gU0NDCmBgYHtyfQptbTAgPC0gbG1lcihMU0NDIH4gVHggKyBQcmV2Q00gKyBET1NDQyArIFRlc3RESU1jYXQyMCArICgxfEZBUk1JRC9Db3dJRCksIGRhdGE9U0RDVENPV0RISSkKZW1tIDwtIGVtbWVhbnMobW0wLCB+VHgpICU+JSB0aWR5KCkKZW1tCmBgYAoKCiMjIyBTdGVwIDZjOiBGaW5hbCBtb2RlbCAodXNpbmcgMTAlIHJ1bGUpIHJlcG9ydGVkIGFzIGJhY2stdHJhbnNmb3JtZWQgZXN0aW1hdGVkIG1hcmdpbmFsIG1lYW5zICh+TFNtZWFucykKR2VvbWV0cmljIG1lYW4gU0NDCmBgYHtyfQptbTAgPC0gbG1lcihMU0NDIH4gVHggKyBQcmV2Q00gKyBET1NDQyArIFRlc3RESU1jYXQyMCArICgxfEZBUk1JRC9Db3dJRCksIGRhdGE9U0RDVENPV0RISSkKZW1tIDwtIGVtbWVhbnMobW0wLCB+VHgpICU+JSB0aWR5KCkKZW1tJFNDQyA8LSBleHAoZW1tJGVzdGltYXRlKSAKZW1tJExDTCA8LSBleHAoZW1tJGFzeW1wLkxDTCkKZW1tJFVDTCA8LSBleHAoZW1tJGFzeW1wLlVDTCkKZW1tIDwtIGVtbSAlPiUgc2VsZWN0KFR4LFNDQyxMQ0wsVUNMKQplbW0KYGBgCjxicj4KPGJyPgo8YnI+CgoKUmVwb3J0ZWQgYXMgYmFjay10cmFuc2Zvcm1lZCBlc3RpbWF0ZWQgbWFyZ2luYWwgbWVhbnMgYnkgaGVyZCB0ZXN0IGRheSwgbm8gaW50ZXJhY3Rpb24gd2l0aCB0ZXN0IERJTQpgYGB7cn0KbW0wIDwtIGxtZXIoTFNDQyB+IFR4ICsgVGVzdERJTWNhdDIwICsgRE9TQ0MgKyBQcmV2Q00gKyAoMXxGQVJNSUQvQ293SUQpLCBkYXRhPVNEQ1RDT1dESEkpCmF0eCA8LSBjKDEwLDMwLDUwLDcwLDkwLDExMCkKZW1tIDwtIGVtbWVhbnMobW0wLCB+VHgqVGVzdERJTWNhdDIwLCBhdD1saXN0KGF0eCkpICU+JSB0aWR5KCkKZW1tJFNDQyA8LSBleHAoZW1tJGVzdGltYXRlKQplbW0kTENMIDwtIGV4cChlbW0kYXN5bXAuTENMKQplbW0kVUNMIDwtIGV4cChlbW0kYXN5bXAuVUNMKQplbW0gPC0gZW1tICU+JSBzZWxlY3QoVHgsVGVzdERJTWNhdDIwLFNDQyxMQ0wsVUNMKQpjdXJ2ZSA8LSBnZ3Bsb3QoZW1tKSArIGNvb3JkX2NhcnRlc2lhbih5bGltID0gKGMoMCwxMDApKSkgKyBhZXMoeD1UZXN0RElNY2F0MjAsIHk9U0NDLCBncm91cD1UeCwgY29sb3VyPVR4KSArIGdlb21fcG9pbnQoKSArIGdlb21fbGluZShhZXMoY29sb3VyPVR4LGxpbmV0eXBlPVR4KSkgKyBnZW9tX3JpYmJvbihhZXMoeW1pbj1lbW0kTENMLCB5bWF4PWVtbSRVQ0wsY29sb3VyPVR4LGZpbGw9VHgpLCBsaW5ldHlwZT0wLCBhbHBoYT0wLjEpCmN1cnZlCmBgYAo8YnI+Cjxicj4KPGJyPgo8YnI+CgpSZXBvcnRlZCBhcyBiYWNrLXRyYW5zZm9ybWVkIGVzdGltYXRlZCBtYXJnaW5hbCBtZWFucyBieSBoZXJkIHRlc3QgZGF5LCB3aXRoIGludGVyYWN0aW9uIHdpdGggdGVzdCBESU0KYGBge3J9Cm1tMCA8LSBsbWVyKExTQ0MgfiBUeCpUZXN0RElNY2F0MjAgKyBQcmV2Q00gKyBET1NDQyArICgxfEZBUk1JRC9Db3dJRCksIGRhdGE9U0RDVENPV0RISSkKYXR4IDwtIGMoMTAsMzAsNTAsNzAsOTAsMTEwKQplbW0gPC0gZW1tZWFucyhtbTAsIH5UeCpUZXN0RElNY2F0MjAsIGF0PWxpc3QoYXR4KSkgJT4lIHRpZHkoKQplbW0kU0NDIDwtIGV4cChlbW0kZXN0aW1hdGUpCmVtbSRMQ0wgPC0gZXhwKGVtbSRhc3ltcC5MQ0wpCmVtbSRVQ0wgPC0gZXhwKGVtbSRhc3ltcC5VQ0wpCmVtbSA8LSBlbW0gJT4lIHNlbGVjdChUeCxUZXN0RElNY2F0MjAsU0NDLExDTCxVQ0wpCmN1cnZlIDwtIGdncGxvdChlbW0pICsgY29vcmRfY2FydGVzaWFuKHlsaW0gPSAoYygwLDEwMCkpKSArIGFlcyh4PVRlc3RESU1jYXQyMCwgeT1TQ0MsIGdyb3VwPVR4LCBjb2xvdXI9VHgpICsgZ2VvbV9wb2ludCgpICsgZ2VvbV9saW5lKGFlcyhjb2xvdXI9VHgsbGluZXR5cGU9VHgpKSArIGdlb21fcmliYm9uKGFlcyh5bWluPWVtbSRMQ0wsIHltYXg9ZW1tJFVDTCxjb2xvdXI9VHgsZmlsbD1UeCksIGxpbmV0eXBlPTAsIGFscGhhPTAuMSkKY3VydmUKYGBgCjxicj4KPGJyPgo8YnI+Cjxicj4KIyMjIFN0ZXAgNzogTW9kZWwgZGlhZ25vc3RpY3MKQ2hlY2tpbmcgaG9tb3NrZWRhc3RpY2l0eSBhc3N1bXB0aW9uICh2YXJpYW5jZSBvZiByZXNpZHVhbHMpCmBgYHtyfQptbTAgPC0gbG1lcihMU0NDIH4gVHggKyBUZXN0RElNY2F0MjAgKyAoMXxGQVJNSUQvQ293SUQpLCBkYXRhPVNEQ1RDT1dESEkpCmdncGxvdChkYXRhLmZyYW1lKGV0YT1wcmVkaWN0KG1tMCx0eXBlPSJsaW5rIikscGVhcnNvbj1yZXNpZHVhbHMobW0wLHR5cGU9InBlYXJzb24iKSksCiAgICAgICBhZXMoeD1ldGEseT1wZWFyc29uKSkgKwogIGdlb21fcG9pbnQoKSArCiAgdGhlbWVfYncoKQpgYGAKCgpDaGVja2luZyBub3JtYWxpdHkgb2YgcmVzaWR1YWxzCmBgYHtyfQpxcW5vcm0ocmVzaWR1YWxzKG1tMCkpCmBgYApFdmlkZW5jZSBvZiBzbWFsbCBMIHRhaWwuIEFsdGhvdWdoIHNvbWUgZXZpZGVuY2Ugb2YgaGV0ZXJvc2tlZGFzY2l0aXR5IGFuZCBub3QgcGVyZmVjdGx5IG5vcm1hbCByZXNpZHVhbHMsIEkgYmVsaWV2ZSB0aGVzZSBhcmUgYWxsb3dhYmxlIGZvciB0aGlzIG1vZGVsLiAKCgojIE91dGNvbWUgNDogTWlsayB5aWVsZAoKCiMjIE1vZGVsbGluZyBwbGFuCk1vZGVsIHR5cGU6IExpbmVhciBtaXhlZCBtb2RlbHMsIHJhbmRvbSBpbnRlcmNlcHRzIGZvciBmYXJtIGFuZCBjb3cgd2lsbCBiZSBmaXR0ZWQgdG8gYWNjb3VudCBmb3IgcmVwZWF0ZWQgbWVhc3VyZXMgd2l0aGluIGNvd3MsIGFuZCBjbHVzdGVyaW5nIG9mIGNvd3Mgd2l0aGluIGhlcmRzCgpTdGVwIDE6IElkZW50aWZ5IHBvdGVudGlhbCBjb25mb3VkZXJzIHVzaW5nIGEgZGlyZWN0ZWQgYWN5Y2xpYyBncmFwaCAoREFHKQoKU3RlcCAyOiBJZGVudGlmeSBjb3JyZWxhdGVkIHZhcmlhYmxlcyB1c2luZyBwZWFyc29uIGFuZCBrZW5kYWxscyBjb3JyZWxhdGlvbiBjb2VmZmljaWVudHMKClN0ZXAgMzogQ3JlYXRlIG1vZGVsIHdpdGggYWxsIHBvdGVudGlhbCBjb25mb3VuZGVycwoKU3RlcCA0OiBJbnZlc3RpZ2F0ZSBwb3RlbnRpYWwgZWZmZWN0IG1lYXN1cmUgbW9kaWZpY2F0aW9uCgpTdGVwIDU6IFJlbW92ZSB1bm5lY2Nlc2FyeSBjb3ZhcmlhdGVzIGluIGJhY2t3YXJkcyBzdGVwd2lzZSBmYXNoaW9uIHVzaW5nIDEwJSBydWxlIChpLmUuIGlmIGVzdGltYXRlZCBkaWZmZXJlbmNlIGluIG1pbGsgeWllbGQgY2hhbmdlcyBieSA+MTAlIGFmdGVyIHJlbW92aW5nIHRoZSBjb3ZhcmlhdGUsIHRoZSBjb3ZhcmlhdGUgaXMgcmV0YWluZWQgaW4gdGhlIG1vZGVsKQoKU3RlcCA2OiBSZXBvcnQgZmluYWwgbW9kZWwKClN0ZXAgNzogTW9kZWwgZGlhZ25vc3RpY3MKCiMjIyBTdGVwIDEgJiAyIElkZW50aWZ5IHBvdGVudGlhbCBjb25mb3VuZGVycyB1c2luZyBhIERBRwpgYGB7ciBmaWcud2lkdGg9MTB9CmxpYnJhcnkoRGlhZ3JhbW1lUikKbWVybWFpZCgiZ3JhcGggTFIKICAgICAgICBUKFRyZWF0bWVudCktLT5VKE1pbGsgeWllbGQpCiAgICAgICAgQShBZ2UpLS0+VAogICAgICAgIFAoUGFyaXR5KS0tPlQKICAgICAgICBNKFlpZWxkIGF0IGRyeS1vZmYpLS0+VAogICAgICAgIFMoU0NDIGR1cmluZyBwcmV2IGxhY3RhdGlvbiktLT5UCiAgICAgICAgQyhDTSBpbiBwcmV2IGxhY3QpLS0+VAogICAgICAgIEQoRGF5cyBpbiBtaWxrIGF0IHRlc3QpLS0+VQogICAgICAgIEQtLT5UCiAgICAgICAgQS0tPlUKICAgICAgICBQLS0+VQogICAgICAgIE0tLT5VCiAgICAgICAgUy0tPlUKICAgICAgICBDLS0+VQogICAgICAgIEMtLT5NCiAgICAgICAgUC0tPkMKICAgICAgICBQLS0+UwogICAgICAgIFAtLT5NCiAgICAgICAgQS0tPlAKICAgICAgICBBLS0+QwogICAgICAgIEEtLT5TCiAgICAgICAgQS0tPk0KICAgICAgICBNLS0+UwogICAgICAgIEMtLT5TCiAgICAgICAgc3R5bGUgQSBmaWxsOiNGRkZGRkYsIHN0cm9rZS13aWR0aDowcHgKICAgICAgICBzdHlsZSBUIGZpbGw6I0ZGRkZGRiwgc3Ryb2tlLXdpZHRoOjJweAogICAgICAgIHN0eWxlIFAgZmlsbDojRkZGRkZGLCBzdHJva2Utd2lkdGg6MHB4CiAgICAgICAgc3R5bGUgTSBmaWxsOiNGRkZGRkYsIHN0cm9rZS13aWR0aDowcHgKICAgICAgICBzdHlsZSBTIGZpbGw6I0ZGRkZGRiwgc3Ryb2tlLXdpZHRoOjBweAogICAgICAgIHN0eWxlIEMgZmlsbDojRkZGRkZGLCBzdHJva2Utd2lkdGg6MHB4CiAgICAgICAgc3R5bGUgSSBmaWxsOiNGRkZGRkYsIHN0cm9rZS13aWR0aDowcHgKICAgICAgICBzdHlsZSBEIGZpbGw6I0ZGRkZGRiwgc3Ryb2tlLXdpZHRoOjBweAogICAgICAgIHN0eWxlIFUgZmlsbDojRkZGRkZGLCBzdHJva2Utd2lkdGg6MnB4CiAgICAgICAgIikKYGBgCkFjY29yZGluZyB0byB0aGlzIERBRywgSSBtYXkgbmVlZCB0byBjb250cm9sIGZvciB0aGUgZm9sbG93aW5nIHZhcmlhYmxlczoKClBhcml0eSBbIlBhcml0eSJdIDwtIEFnZSBub3Qgb2ZmZXJlZCBhcyBoaWdobHkgY29ycmVsYXRlZAoKWWllbGQgYXQgbW9zdCByZWNlbnQgdGVzdCBiZWZvcmUgZHJ5IG9mZiBbIkRPTVkiXQoKU29tYXRpYyBjZWxsIGNvdW50IGF0IGxhc3QgaGVyZCB0ZXN0IGR1cmluZyBwcmV2aW91cyBsYWN0YXRpb24gWyJET1NDQyJdIDwtIFByZXZTQ0NISSBub3Qgb2ZmZXJlZCBhcyBjb3JyZWxhdGVkCgpDbGluaWNhbCBtYXN0aXRpcyBpbiBwcmV2aW91cyBsYWN0YXRpb24gWyJQcmV2Q00iXQoKRGF5cyBpbiBtaWxrIGF0IGhlcmQgdGVzdCAoY2F0ZWdvcnksIDAtMjAgIjEwIiwgMjEtNDAgIjMwIiBldGMpIFsiVGVzdERJTWNhdDIwIl0KCjxicj4KPGJyPgo8YnI+Cjxicj4KCiMjIyBTdGVwIDM6IENyZWF0ZSBhIG1vZGVsIHdpdGggYWxsIHBvdGVudGlhbCBjb3ZhcmlhdGVzCgpgYGB7cn0KbW0wIDwtIGxtZXIoTVkgfiBUeCArIFRlc3RESU1jYXQyMCArIFBhcml0eSArIERPU0NDICsgUHJldkNNICsgRE9NWSArICgxfEZBUk1JRC9Db3dJRCksIGRhdGE9U0RDVENPV0RISSkKc3VtbWFyeShtbTApCmNhcjo6dmlmKG1tMCkKYGBgCjxicj4KPGJyPgo8YnI+Cjxicj4KCiMjIyBTdGVwIDQ6IEludmVzdGlnYXRlIGVmZmVjdCBtZWFzdXJlIG1vZGlmaWNhdGlvbgoKIyMjIFR4OkZBUk0gKFAgPCAwLjA1KQpgYGB7cn0KbW0wIDwtIGxtZXIoTVkgfiBUeCpGQVJNSUQgKyBQYXJpdHkgKyBUZXN0RElNY2F0MjAgKyBET1NDQyArIERPTVkgKyBQcmV2Q00gKyAoMXxDb3dJRCksIGRhdGE9U0RDVENPV0RISSkKY2FyOjpBbm92YShtbTApCmBgYAoKU2lnbmlmaWNhbnQgaW50ZXJhY3Rpb24gdGVybS4KCkRlc2Npc2lvbjogUmV2aXNpdCB0aGlzIGFmdGVyIGNvdmFyaWF0ZXMgYXJlIGZpbmFsaXplZC4KPGJyPgo8YnI+Cjxicj4KPGJyPgpUeDpESU0gY2F0ZWdvcnkgYXQgaGVyZCB0ZXN0IChQID4gMC4wNSkKYGBge3J9Cm1tMCA8LSBsbWVyKE1ZIH4gVHgqVGVzdERJTWNhdDIwICsgUGFyaXR5ICsgRE9TQ0MgKyBET01ZICsgUHJldkNNICsgKDF8RkFSTUlEL0Nvd0lEKSwgZGF0YT1TRENUQ09XREhJKQpjYXI6OkFub3ZhKG1tMCkKYGBgCjxicj4KPGJyPgo8YnI+Cjxicj4KVHg6UGFyaXR5IChQID4gMC4wNSkKYGBge3J9Cm1tMCA8LSBsbWVyKE1ZIH4gVHgqUGFyaXR5ICsgVGVzdERJTWNhdDIwICsgRE9TQ0MgKyBET01ZICsgUHJldkNNICsgKDF8RkFSTUlEL0Nvd0lEKSwgZGF0YT1TRENUQ09XREhJKQpjYXI6OkFub3ZhKG1tMCkKYGBgCjxicj4KPGJyPgo8YnI+Cjxicj4KVHg6UHJldkNNIChQID4gMC4wNSkKYGBge3J9Cm1tMCA8LSBsbWVyKE1ZIH4gVHgqUHJldkNNICsgUGFyaXR5ICsgVGVzdERJTWNhdDIwICsgRE9TQ0MgKyBET01ZICsgKDF8RkFSTUlEL0Nvd0lEKSwgZGF0YT1TRENUQ09XREhJKQpjYXI6OkFub3ZhKG1tMCkKYGBgCjxicj4KPGJyPgo8YnI+Cjxicj4KCiMjIyBTdGVwIDU6IFJlbW92ZSB1bm5lY2Vzc2FyeSBjb3ZhcmlhdGVzIHVzaW5nIGJhY2t3YXJkcyBzZWxlY3Rpb24gKDEwJSBydWxlKQoKVGhlIG9yZGVyIGZvciByZW1vdmluZyBjb3ZhcmlhdGVzIHdpbGwgYmUgaW4gaW5jcmVhc2luZyBsaWtlbGlob29kIG9mIGJlaW5nIGEgY29uZm91bmRlciwgd2hpY2ggaXMgYmFzZWQgb24gbXkga25vd2xlZGdlIGFib3V0IHRoZSB2YXJpYWJsZXMgYW5kIHRoZWlyIGRpc3RyaWJ1dGlvbiBpbiB0cmVhdG1lbnQgZ3JvdXBzLgoKVGhpcyBvcmRlciB3aWxsIGJlOiBET1NDQywgUHJldkNNLCBQYXJpdHksIERPTVkKClRlc3RESU1jYXQyMCB3aWxsIG5vdCBiZSByZW1vdmVkIChmb3JjZWQgaW50byBtb2RlbCkuIAoKIAoKClN0ZXAgNWE6IEZ1bGwgbW9kZWwKYGBge3J9Cm1tMCA8LSBsbWVyKE1ZIH4gVHggKyBQcmV2Q00gKyBQYXJpdHkgKyBUZXN0RElNY2F0MjAgKyBET1NDQyArIERPTVkgKyAoMXxGQVJNSUQvQ293SUQpLCBkYXRhPVNEQ1RDT1dESEkpIAptbTAgJT4lIHRpZHkoKSAlPiUgc2VsZWN0KHRlcm0sZXN0aW1hdGUpCmBgYAo8YnI+Cjxicj4KPGJyPgo8YnI+ClN0ZXAgNWI6IFJlbW92aW5nIERPU0NDCmBgYHtyfQptbTAgPC0gbG1lcihNWSB+IFR4ICsgUHJldkNNICsgUGFyaXR5ICsgVGVzdERJTWNhdDIwICsgRE9NWSArICgxfEZBUk1JRC9Db3dJRCksIGRhdGE9U0RDVENPV0RISSkgCm1tMCAlPiUgdGlkeSgpICU+JSBzZWxlY3QodGVybSxlc3RpbWF0ZSkKYGBgCkNoYW5nZWQgYnkgPDEwJS4gRE9TQ0Mgc3RheXMgb3V0Cjxicj4KPGJyPgo8YnI+Cjxicj4KClN0ZXAgNWM6IHJlbW92aW5nIFByZXZDTQpgYGB7cn0KbW0wIDwtIGxtZXIoTVkgfiBUeCArIFBhcml0eSArIFRlc3RESU1jYXQyMCArIERPTVkgKyAoMXxGQVJNSUQvQ293SUQpLCBkYXRhPVNEQ1RDT1dESEkpICAKbW0wICU+JSB0aWR5KCkgJT4lIHNlbGVjdCh0ZXJtLGVzdGltYXRlKQpgYGAKQ2hhbmdlZCBieSA8IDEwJS4gUHJldkNNIHN0YXlzIG91dC4gCgo8YnI+Cjxicj4KPGJyPgo8YnI+ClN0ZXAgNWQ6IHJlbW92aW5nIFBhcml0eQpgYGB7cn0KbW0wIDwtIGxtZXIoTVkgfiBUeCArIFRlc3RESU1jYXQyMCArIERPTVkgKyAoMXxGQVJNSUQvQ293SUQpLCBkYXRhPVNEQ1RDT1dESEkpIAptbTAgJT4lIHRpZHkoKSAlPiUgc2VsZWN0KHRlcm0sZXN0aW1hdGUpCmBgYApDaGFuZ2VkIGJ5ID4xMCUuICBQYXJpdHkgc3RheXMgaW4uCjxicj4KPGJyPgo8YnI+CgpTdGVwIDVlOiByZW1vdmluZyBET01ZCmBgYHtyfQptbTAgPC0gbG1lcihNWSB+IFR4ICsgUGFyaXR5ICsgVGVzdERJTWNhdDIwICsgKDF8RkFSTUlEL0Nvd0lEKSwgZGF0YT1TRENUQ09XREhJKSAKbW0wICU+JSB0aWR5KCkgJT4lIHNlbGVjdCh0ZXJtLGVzdGltYXRlKQpgYGAKQ2hhbmdlZCBieSA+MTAlLiAgRE9NWSBzdGF5cyBpbi4gCjxicj4KPGJyPgo8YnI+Cjxicj4KCiMjIyBSZXZpc2l0IGVmZmVjdCBtZWFzdXJlIG1vZGlmaWNhdGlvbiBwcmV2aW91c2x5IGlkZW50aWZpZWQKRWZmZWN0IG1lYXN1cmUgbW9kaWZpY2F0aW9uIHdpdGggaGVyZCAoUCA8IDAuMDUpCmBgYHtyfQptbTAgPC0gbG1lcihNWSB+IFR4KkZBUk1JRCArIFRlc3RESU1jYXQyMCArIFBhcml0eSArIERPTVkgKyAoMXxDb3dJRCksIGRhdGE9U0RDVENPV0RISSkKY2FyOjpBbm92YShtbTApCmBgYAoKRGVjaXNpb24gLSB3aWxsIGludmVzdGlnYXRlIGVmZmVjdCBlc3RpbWF0ZXMgc3RyYXRpZmllZCBieSBoZXJkCjxicj4KPGJyPgo8YnI+Cjxicj4KCiMjIyBTdGVwIDZhOiBGaW5hbCBtb2RlbCBmb3IgbWlsayB5aWVsZCAoMSAtIDEyMCBESU0pLCBzdHJhdGlmaWVkIGJ5IGhlcmQuIApNb2RlbCBvdXRwdXQKYGBge3J9Cm1tMCA8LSBsbWVyKE1ZIH4gVHgqRkFSTUlEICsgVGVzdERJTWNhdDIwICsgUGFyaXR5ICsgRE9NWSArICgxfENvd0lEKSwgZGF0YT1TRENUQ09XREhJKQplbW1lYW5zKG1tMCxwYWlyd2lzZSB+IFR4IHwgRkFSTUlEKQpgYGAKSXQgYXBwZWFycyB0aGF0IGhlcmQgMiAoODcgY293cykgYW5kIGhlcmQgNiAoNDIgY293cykgaGF2ZSBleHRyZW1lIHJlc3VsdHMgY29tcGFyZWQgd2l0aCB0aGUgb3RoZXIgaGVyZHMuICBJbiBoZXJkIDIsIEJEQ1QgaGFkIG11Y2ggaGlnaGVyIE1ZIHRoYW4gU0RDVCBjb3dzLiAgSW4gaGVyZCA2LCBpdCB3YXMgdGhlIG9wcG9zaXRlIGVmZmVjdC4gIEdpdmVuIHRoZXNlIHdlcmUgdGhlIHNtYWxsZXN0IGhlcmRzIGluIHRoZSBzdHVkeSwgSSB3aWxsIHJlcG9ydCBhIHBvb2xlZCByZXN1bHQuIAo8YnI+Cjxicj4KPGJyPgo8YnI+CgojIyMgU3RlcCA2Ymk6IEZpbmFsIG1vZGVsIHJlcG9ydGVkIHdpdGhvdXQgaGVyZCBpbnRlcmFjdGlvbgpNb2RlbCBvdXRwdXQKYGBge3J9CnN1bW1hcnkobG1lcihNWSB+IFR4ICsgVGVzdERJTWNhdDIwICsgUGFyaXR5ICsgRE9NWSsgKDF8RkFSTUlEL0Nvd0lEKSwgZGF0YT1TRENUQ09XREhJKSkKYGBgCgpNb2RlbCBvdXRwdXQgKHRpZHkpCmBgYHtyfQpsbWVyKE1ZIH4gVHggKyBUZXN0RElNY2F0MjAgKyBQYXJpdHkgKyBET01ZICsgKDF8RkFSTUlEL0Nvd0lEKSwgZGF0YT1TRENUQ09XREhJKSAlPiUgdGlkeShjb25mLmludD1UUlVFKQpgYGAKCiMjIyMgSUNDIGZvciBNWSAoMSAtIDEyMCBESU0pCmBgYHtyfQptbTAgPC0gbG1lcihNWSB+IDEgKyAoMXxGQVJNSUQvQ293SUQpLCBkYXRhPVNEQ1RDT1dESEkpCnN1bW1hcnkobW0wKQpgYGAKCklDQyAoQ293SUQpID0gMC4yNQpJQ0MgKEZBUk1JRCkgPSAwLjA5CgoKRW1tZWFucyB3aXRob3V0IFRlc3RESU0gaW50ZXJhY3Rpb24KYGBge3J9Cm1tMCA8LSBsbWVyKE1ZIH4gVHggKyBUZXN0RElNY2F0MjAgKyBQYXJpdHkgKyBET01ZKyAoMXxGQVJNSUQvQ293SUQpLCBkYXRhPVNEQ1RDT1dESEkpCgplbW1lYW5zKG1tMCx+VHgpICU+JSB0aWR5CmBgYAoKCiMjIyBTdGVwIDZiaWk6IEZpbmFsIG1vZGVsIHJlcG9ydGVkIHdpdGhvdXQgaGVyZCBpbnRlcmFjdGlvbiAodXNpbmcgUC12YWx1ZSBiYXNlZCBiYWNrd2FyZHMgZWxpbWluYXRpb24pCk1vZGVsIG91dHB1dApgYGB7cn0Kc3VtbWFyeShsbWVyKE1ZIH4gVHggKyBUZXN0RElNY2F0MjAgKyBQYXJpdHkgKyAoMXxGQVJNSUQvQ293SUQpLCBkYXRhPVNEQ1RDT1dESEkpKQpgYGAKCgo8YnI+Cjxicj4KPGJyPgo8YnI+CgojIyMgU3RlcCA2YmlpaTogRmluYWwgbW9kZWwgKHVzaW5nIDEwJSBydWxlKSByZXBvcnRlZCBhcyBlc3RpbWF0ZWQgbWFyZ2luYWwgbWVhbnMgYnkgdGVzdCBESU0gY2F0ZWdvcnkgKG5vIGludGVyYWN0aW9uIHdpdGggdGVzdCBESU0pCmBgYHtyfQptbTAgPC0gbG1lcihNWSB+IFR4ICsgVGVzdERJTWNhdDIwICsgUGFyaXR5ICsgRE9NWSArICgxfEZBUk1JRC9Db3dJRCksIGRhdGE9U0RDVENPV0RISSkKCmF0eCA8LSBjKDEwLDMwLDUwLDcwLDkwLDExMCkKZW1tIDwtIGVtbWVhbnMobW0wLCB+VHgqVGVzdERJTWNhdDIwKSAlPiUgdGlkeSgpCmVtbSRNWSA8LSBlbW0kZXN0aW1hdGUKZW1tJExDTCA8LSBlbW0kYXN5bXAuTENMCmVtbSRVQ0wgPC0gZW1tJGFzeW1wLlVDTAplbW0gJT4lIHNlbGVjdChUeCxUZXN0RElNY2F0MjAsTVksTENMLFVDTCkKYGBgCgpQbG90dGluZyBtb2RlbCBhcyBwcmVkaWN0ZWQgdmFsdWVzCmBgYHtyfQpjdXJ2ZSA8LSBnZ3Bsb3QoZW1tKSArIGNvb3JkX2NhcnRlc2lhbih5bGltID0gKGMoMCw1NSkpKSArIGFlcyh4PVRlc3RESU1jYXQyMCwgeT1NWSwgZ3JvdXA9VHgsIGNvbG91cj1UeCkgKyBnZW9tX3BvaW50KCkgKyBnZW9tX2xpbmUoYWVzKGNvbG91cj1UeCxsaW5ldHlwZT1UeCkpICsgZ2VvbV9yaWJib24oYWVzKHltaW49ZW1tJExDTCwgeW1heD1lbW0kVUNMLGNvbG91cj1UeCxmaWxsPVR4KSwgbGluZXR5cGU9MCwgYWxwaGE9MC4xKQpjdXJ2ZQpgYGAKPGJyPgo8YnI+Cjxicj4KPGJyPgpSZXBvcnRlZCBhcyBlc3RpbWF0ZWQgbWFyZ2luYWwgbWVhbnMgYnkgdGVzdCBkYXkgY2F0ZWdvcnkgd2l0aCBpbnRlcmFjdGlvbiB3aXRoIGhlcmQgdGVzdCBESU0KYGBge3J9Cm1tMCA8LSBsbWVyKE1ZIH4gVHgqVGVzdERJTWNhdDIwICsgUGFyaXR5ICsgRE9NWSArICgxfEZBUk1JRC9Db3dJRCksIGRhdGE9U0RDVENPV0RISSkKYXR4IDwtIGMoMTAsMzAsNTAsNzAsOTAsMTEwKQplbW0gPC0gZW1tZWFucyhtbTAsIH5UeCpUZXN0RElNY2F0MjAsIGF0PWxpc3QoYXR4KSkgJT4lIHRpZHkoKQplbW0kTVkgPC0gZW1tJGVzdGltYXRlCmVtbSRMQ0wgPC0gZW1tJGFzeW1wLkxDTAplbW0kVUNMIDwtIGVtbSRhc3ltcC5VQ0wKZW1tIDwtIGVtbSAlPiUgc2VsZWN0KFR4LFRlc3RESU1jYXQyMCxNWSxMQ0wsVUNMKQoKY3VydmUgPC0gZ2dwbG90KGVtbSkgKyBjb29yZF9jYXJ0ZXNpYW4oeWxpbSA9IChjKDAsNTUpKSkgKyBhZXMoeD1UZXN0RElNY2F0MjAsIHk9TVksIGdyb3VwPVR4LCBjb2xvdXI9VHgpICsgZ2VvbV9wb2ludCgpICsgZ2VvbV9saW5lKGFlcyhjb2xvdXI9VHgsbGluZXR5cGU9VHgpKSArIGdlb21fcmliYm9uKGFlcyh5bWluPWVtbSRMQ0wsIHltYXg9ZW1tJFVDTCxjb2xvdXI9VHgsZmlsbD1UeCksIGxpbmV0eXBlPTAsIGFscGhhPTAuMSkKY3VydmUKYGBgCgojIyMgU3RlcCA3OiBNb2RlbCBkaWFnbm9zdGljcwpDaGVja2luZyBob21vc2tlZGFzdGljaXR5IGFzc3VtcHRpb24gKHZhcmlhbmNlIG9mIHJlc2lkdWFscykKYGBge3J9Cm1tMCA8LSBsbWVyKE1ZIH4gVHggKyBUZXN0RElNY2F0MjAgKyBQYXJpdHkgK0RPTVkrICgxfEZBUk1JRC9Db3dJRCksIGRhdGE9U0RDVENPV0RISSkKZ2dwbG90KGRhdGEuZnJhbWUoZXRhPXByZWRpY3QobW0wLHR5cGU9ImxpbmsiKSxwZWFyc29uPXJlc2lkdWFscyhtbTAsdHlwZT0icGVhcnNvbiIpKSwKICAgICAgIGFlcyh4PWV0YSx5PXBlYXJzb24pKSArCiAgZ2VvbV9wb2ludCgpICsKICB0aGVtZV9idygpCmBgYAoKQ2hlY2tpbmcgbm9ybWFsaXR5IG9mIHJlc2lkdWFscwpgYGB7cn0KcXFub3JtKHJlc2lkdWFscyhtbTApKQpgYGAKCkkgYW0gaGFwcHkgd2l0aCBob21vc2NlZGFzdGljaXR5IGFuZCBub3JtYWwgcmVzaWR1YWxzIGFzc3VtcHRpb25zCgo=